This week I continued working on the perception subsystem for the TrashDash project. I focused on setting up the camera pipeline and running MediaPipe for real-time hand detection. I was able to successfully run the MediaPipe hand tracking model and verify that the system can detect hand landmarks from the camera stream.
In addition to the vision pipeline, I also started testing the Time-of-Flight (ToF) sensor (VL53L1X) using Arduino. I worked on setting up the sensor and running example code to read distance measurements. While I was able to initialize the sensor and communicate with it, I encountered some instability in getting consistent distance outputs, which I suspect is related to hardware connection issues. I am currently debugging the wiring and setup to ensure reliable readings.
One challenge this week is ensuring stable and accurate perception data from both the camera and ToF sensor. For the vision subsystem, lighting conditions and hand poses can affect detection quality. For the ToF sensor, hardware reliability and communication stability need to be improved before integration.
Next week, I plan to continue improving the robustness of the MediaPipe hand detection pipeline and begin integrating the 3D hand position into the control logic. I will also fix the ToF sensor setup and verify that it can provide consistent distance measurements for obstacle detection.