I spent this week cleaning up the system’s Bluetooth module, determining the one-way latency of our wireless data transmission, and establishing a consistent threshold for the incoming accelerometer values on the host device.
To obtain latency metrics, I chose to implement a Round Trip Time (RTT) test. The strategy was to take an initial timestamp on the ESP with the system clock, update the server characteristic and notify the client, wait for a response by observing a change in the server entry, and take the time difference. This came with a few minor issues to be resolved: first, I observed that the characteristic updates were inconsistent and the test resulted in significantly different output values across runs. This was due to the client updating the same buffer as the ESP32 during its response, thus introducing concurrency issues when the devices attempted to update the characteristic simultaneously. I fixed this by separating the transmission and reception to two distinct characteristics, allowing for continuous processing on both sides. Once this was resolved, I noticed that the resulting delay was still too high–around 100ms. After searching online, I came across this article, stating that the default connection interval for the ESP32 ranges from 7.5ms up to as much as 4s: https://docs.espressif.com/projects/esp-idf/en/release-v5.2/esp32c6/api-guides/ble/get-started/ble-connection.html. Having this variance was unacceptable for our purposes, and so I made use of the esp_gap_ble_api library to manually set the maximum connection interval to 20ms. This change greatly reduced the final delay of the test, but having the shorter connection interval means I’ll have to be aware of interference as we integrate a second microcontroller on the 2.4GHz band. The final value of my testing procedure landed our one-way latency at around 40ms, but my belief is that the actual value is even less; this is because of the inherent overhead introduced across the testing code–the operations of looping in the arduino firmware, polling for the client response, and unpacking data all contribute a nonzero latency to the result. Hence, I tested the implementation qualitatively by manually setting a fixed accelerometer threshold and printing over USB on valid spikes. This test produced favorable results, suggesting that the latency could certainly be under 40ms. I was also able to determine an appropriate threshold value for data processing while doing this, which I concluded to be 10 m/s2. This value achieved a reasonable hit detection rate, but we may choose to store multiple thresholds corresponding to different surfaces if the user wishes to play with a uniform actuation force across all surface types. Ultimately, these tests were helpful in our planning towards a low-latency solution, and I believe I’m still on track with the team’s schedule.
In this upcoming week, I plan to move my Bluetooth code into the system controller and assist Ben with audio buffer delay. Specifically, I will:
- Create a functional controller to detect accelerometer hits and play specified audio files before introducing CV.
- Explore ways to minimize audio output latency as much as possible, such as diving into the PyAudio stack, finding a different library, or considering the MIDI controller route suggested to us by Professor Bain.