This week, I worked with Jeffrey to integrate the smartphone app with the Flask backend. After integrating the submodules, we now can play sounds on the Javascript frontend, with notes controlled by the smartphone. However, the latency initially was very slow and jittery, averaging around 150ms but very inconsistent. We discovered that the source of the latency comes from the socket connection between the smartphone and the Flask server, and tried several different socketing methods to try and resolve this issue.
The best solution we’ve found so far was to use an interval-based socketing mechanism, sending 30 updates/sec =~ 33ms between each update. This reduced the latency significantly, but the jitter is still an issue, and we are still unsure where the root of the problem lies (might be from internet load from other applications/people in my apartment?).
Besides that, I also added functionality in Tone.js to activate/deactivate effects and filters. Next week, we will work on integrating octave shifts and pitch shifts using the touchpad.
