The main task of this week was getting audio inputs to work at some baseline level. Caleb and I worked on getting audio inputs registered. While trying to get this working, I discovered alternatives for accessing or controlling headphone jacks we could use in the future like ALSA utilities and jackaudio.org. This week, we used PyAudio in combination with this code from Google to run an ML model for voice recognition. The main purpose of this was to test whether the mic was being used as the audio input source and whether it was picking up sound at a high enough resolution for the ML model to process. View video of demo. Next steps in audio involve getting chroma vectors and note onsets from recordings of me playing violin and displaying that in real time.
I also created a GitHub repository for the team’s code. It currently just hosts frontend code. Next steps involve integration with the audio code.
My progress is on schedule, and I will continue to keep working hard to stay on track. Some deliverables for the upcoming week are creating audio samples for testing the microphone some more with violin. I also want to keep testing how the note onsets are detected for the violin and calculate thresholds for onset detection. I also want to keep working on implementing the frontend to optimize for the user experience.