This week, I made a lot of progress. I continued working on the audio alignment algorithm with Rohan and Caleb. I also conducted several rounds of tests. We continued to test the audio alignment subsystem and the eye tracking systems separately.
Thanks to Professor Jim Bain, we also discovered the issue that was hampering our audio reception. We continued to run into the harmonic problem where the mic picks up audios at 1.5x the frequency of the actual note. We tested our system on Prof. Bain’s vocals for the first time and discovered that the harmonic problem was unique to the resonant violin and decided to dampen the sound with a mute. This solution, however, is imperfect. In the future, I would like to buy a practice mute so I can more evenly dampen all the strings instead of prioritizing only two strings to dampen with my current mute.
There were some changes and attempts made in the audio alignment code as well to get better accuracy with our cursor. We considered implementing rest tracking (rests are musical structures without any frequency being generated, in other words, silence). We ultimately decided against this so users would only get a moving cursor as they’re playing sections with mostly notes.
Finally, we took measurements for latency in our system again and got an updated value even lower than our presented value of 158ms. The librosa.load calls were being made in the wrong place, and therefore were getting included in our latency calculations. Now that we removed these extra calls, the system is aligning audio segments of 50 frames in under 20ms.
I also worked hard on the Final Report (stay tuned!). That’s coming along nicely, I’m just waiting on the results section so we can add the most updated results there as we are continuing to optimize the algorithms up until the end of this week. Overall, our project is almost complete and I am so excited to present it at the demos we have this week!