This week I integrated the audio alignment into the backend and tried numerous different methods to improve audio quality. The initial problem that we had was that instead of detecting the actual note being played, we were detecting the note that is a 5th above or 1.5x the frequency. We attempted to fix this by changing the pvrecorder API that recorded the audio to increase the sampling rate and achieve better audio quality. However, to do this, it required rebuilding the API and creating a new dll file. In trying to do this, we accidently changed some key computer settings on my laptop and wiped the drive that it was running off of. Therefore, we shifted the entire project to Rohan’s laptop and setup all the necessary environments all over again. After making this shift, instead of trying to change the pvrecorder API, we instead tested different locations for the microphone and found a location right near the bridge of the violin that greatly improved the audio quality. Although this location still gives the 5th occasionally, it does so far less frequently than other locations we tested. For instance, we tested right on the player’s shirt collar, however, any movement by the player caused the microphone to move and to pick up a static scraping sound as the microphone brushed against the fabric.
Another problem came in the implementation of the backend. It turns out, in Django, no multiprocessing in Python can be used. This was not known until the entire backend was parallelized. This parallelism was very important for hitting timing use-case requirements and may now only be doable in a subprocess call instead of directly in the Django backend. There is also another api called celery that may make this doable, but we are still unfamiliar with this api.
This upcoming week, I will be looking at the robustness of the MIDI align algorithm as well as how to implement it efficiently in the backend. To do this, we will continue to test with Sanjana playing and see if we can accurately place the cursor in roughly the correct position and if the other audio alignment algorithms such as DTW will improve the MIDI align algorithm.
We are currently still on track and will continue to work hard to stay on track.