This week, I met with Grace to test the audio segmentation algorithm she wrote. We tested it on a sample of Twinkle Twinkle Little Star, as well as Ten Little Monkeys. We found that for each of the two samples, we needed to adjust the RMS threshold to account for differences in the maximum amplitude of the signal; as a result, we realized that we will need to add some way to either standardize the amplitude of our signal or dynamically change the RMS threshold based on the signal’s amplitude.
I also worked to integrate our preprocessing and audio segmentation code all together. Our current pipeline can be found on this GitHub (Segmentation/seg.py for note segmentation, and Pitch Detection/pitch.py for pitch detection) along with some of our past experimentation code.
Furthermore, now that we have audio segmentation, I was able to get pitch detection to work, at least on Twinkle Twinkle. To do so, I used FFT and comb filtering to find and map the fundamental frequency to the MIDI note. I plan to test the pitch detection on more audio samples next week and work with Grace and Deeya to integrate all the stages of our project that we have implemented so far (web app and triggering the preprocessing/audio segmentation/pitch detection pipeline).