This week I worked together with my teammates on system integration and testing. All components were assembled into a complete working pipeline, including adding hand localization logic to the gesture detection module so that the robot actively moves toward the user’s hand position rather than executing a fixed motion sequence. As well as adding the function of first finding the user’s position by turning around. The full end-to-end flow was completed and verified as a unified system. I also helped conduct testing on all individual parts and the overall system. Unit-level tests were performed on the voice recognition module, the OAK-D camera gesture recognition, and the ToF obstacle avoidance, each verified independently before being tested together. The end-to-end tests were then run on the complete pipeline to validate the full interaction flow. Test data was collected and analyzed to evaluate whether the current system’s performance meets the defined user requirements.
In the process of working on this project, since my background is primarily in software engineering and computer systems, it required me to learn a significant amount of knowledge about hardware integration and low-level hardware communication. For example, I learned a lot about RPi usage, including how to set it up from scratch, configure the operating system, and correctly use its GPIO pins for connecting sensors and other components. This was quite different from typical software-only development and required understanding both the physical layout and electrical considerations. I also learned a lot about Ros2 middleware (publish/subscribe model), which was essential for implementing features like voice activation, gesture detection, and navigation. In terms of learning strategies, I found watching online videos and tutorials very useful. Also, reading the official instruction documents is also useful for setting up the different modules that we use, and hands-on experimentation is also helpful for making concepts clear.