This week I worked on improving the strumming algorithm and adding functionality to our glove. I implemented stopping the sounds when the touch sensor has been opened, so users can control how long a sound continues to play. I also added functionality to make the delay controlled by the speed at which the user is moving their hand. The speed estimates are still not very accurate, and we think more filtering and tuning is required for this to work well.
I also conducted latency experiments for our bluetooth communication scheme. The latency of sending a BLE message from the esp32 to the python program is around 30ms, and I used a combination of round trip times samples to get this value. Overall, we also thought about how to design experiments for strum detection accuracy. Now that the pipeline is all connected together, we need to polish everything.
New skills or tools I’ve learned to accomplish capstone tasks:
I needed to learn several new technical skills and knowledge areas.
I learned how to code for the Arduino platform, including reading sensor data and implementing algorithms for strumming detection. I also needed to understand Bluetooth communication, specifically how to establish a connection and send data between a Python program and an ESP32 microcontroller. Additionally, I learned how to use the MediaPipe libraries for motion tracking, which involved understanding how to process and interpret the data from these libraries to integrate with my project.
To acquire this knowledge, I used a combination of strategies. I read documentation for the hardware and software tools, which helped me understand the APIs and limitations of the sensors and microcontroller. I also watched tutorial videos online, which were especially helpful for seeing practical implementations and debugging strategies. Finally, I used AI tools to clarify concepts and suggest solutions when I encountered coding challenges. Combining these strategies allowed me to learn efficiently and apply the knowledge directly to design, implement, and debug my project.