This week I started working on the computer vision subsystem for the TrashDash project. I set up the camera pipeline and began experimenting with MediaPipe for hand detection. The goal of this subsystem is to detect the user’s hand in real time and estimate its position so that the robot can navigate toward the user for trash collection. I installed the necessary libraries on my computer and ran initial tests to verify that MediaPipe can successfully detect hand landmarks from the camera stream.
In addition to basic hand detection, I also started exploring how to combine the RGB image with depth information from the camera in order to estimate the 3D position of the user’s hand. This information will later be used by the control logic to compute the robot’s motion commands. Initial tests confirmed that the camera can stream frames reliably and that MediaPipe can detect the hand landmarks in real time.
One challenge I am currently working on is ensuring stable detection and reducing noise in the hand position estimation. Variations in lighting conditions and hand poses can sometimes affect the detection confidence. To address this, I plan to experiment with filtering and parameter tuning in the vision pipeline.
Next week I plan to continue improving the vision subsystem and begin integrating the hand position output with the control logic.