This week, I integrated the rhythm analysis algorithm with Yuhe and Lucas’s game code. I also experimented with another method of determining the global threshold for onset strengths above which a timestamp should be counted as a note. This one used median and median absolute deviation instead of mean and standard deviation. Theoretically it would have less errors due to any outliers because the difference does not get exponentiated. This method performs similarly to the current method but has slightly more false positives at slower tempos. I also further tested the sliding window method. This one had an even higher amount of false positives at slow tempos. I believe this might be ameliorated by have the window slide more continuously rather than jumping from the first 100 frames to the second 100 frames, for example. The issue with this is that it would increase the audio processing latency, which we want to avoid. I think the method using standard deviation (in blue below) is still the best method overall.
Next week, I plan to help with integration testing and also conduct some user testing to validate that the project meets the user requirements. I also plan to help improve the visual design of the game as needed.
I learned a several new tools while working on this project. At the beginning, I learned the basics of Unity before we pivoted to Yuhe’s self-made lightweight game engine. I found it easiest to learn Unity as a beginner by watching and following along with a Youtube tutorial. By consulting documentation, I got more familiar with the numpy library, especially using it for complex plots. I learned how to use threading in Python, looking at examples on Stack Overflow, in order to create a verification program that could play a song and animate its detected notes at the same time. I also learned how to use MuseScore to compose pieces to use as tests for the rhythm analysis. For this I was able to mostly teach myself and occasionally Google any features I couldn’t find on my own.