This week I worked more on the system’s Bluetooth component and gathered verification metrics in preparation for our final presentation. One of the team’s design requirements is to ensure a packet loss under 2% for the BLE, so I performed a packet counting test between the MCUs and the laptop. The strategy was to have a counter incremented before notifying the central device of incoming data, and conversely have a separate counter to be incremented upon entry to that ESP’s corresponding notification handler. I ran this test for five minutes with the two sticks communicating simultaneously, and by taking the difference of the two counters I came to a packet loss of 2.01 percent (26,815 packets received vs. 27,364 packets sent). This was a surprisingly high data loss for our use case, leading me to confirm that our issues with latency were most likely stemming from the wireless transmission. Looking back at my firmware implementation, the latent drumstick would delay for a few seconds, then output a stream of data like this:
This was ultimately a hint that the ESP was forced to retransmit as well as queue packets across multiple connection intervals. After reading more about the Bluetooth stack, I realized that a fixed connection interval of 7.5ms was too short to allow the central device to schedule events, therefore resulting in packet collisions between the two boards. I also found that sending multiple notifications to the laptop as quickly as possible would overwhelm the event processing queue and cause it to fall behind in timing (similar to our struggles with the audio output). The solution was to raise the connection intervals to 20ms to allow for more schedulability between the devices, and to also raise the notification rates from 1ms up to 21 and 23ms, staggering them to further prevent queue congestion. This led to a much smoother response between the two drumsticks, and the safer approach did not seem to have a noticeable impact on performance.
One skill I’ve picked up while working on this capstone project is quickly reading through online documentation for relevant information. In order to make our Bluetooth, OpenCV and multithreaded audio modules cooperate, I’ve read everything from online tutorials by Nordic Semiconductor, web articles on Geeksforgeeks, and pure datasheets for our microcontrollers while problem solving. I’ve also learned to take as much input as possible from people with experience, such as the teaching staff and faculty, which has made the setbacks we’ve encountered much more manageable.
This week, I plan to help further optimize the system’s CV and Bluetooth processing. The problems we currently face are the HSV lighting inconsistencies along with a dip in performance when drumsticks are in view of the camera. I believe we’re still on track with our schedule, although we may be approaching significant design tradeoff decisions to be able to bring down the response time.