Last week, we had a successful interim demo where we had our transcription pipeline working for a recording of Twinkle Twinkle Little Star. We also met with a flutist from the School of Music to get her feedback on our pipeline and obtain some sample recordings. She found the interface intuitive and easy-to-use, though we did run into some bugs with audio file formats and found that our note segmentation struggled a little with slurred notes.
This week, we focused on the following items:
- Fix the denoising step
- Set up websockets for real-time BPM adjustment
- Any remaining frontend-backend integration remaining for the web app. (for e.g., earlier we had some bugs with audio file formats and with recording audio directly via the web app)
- Using a Short Time Energy approach instead of RMS to perform note segmentation. This helped to better account for rests/slurs in the music.
Later this weekend, we are meeting again with another flutist from the SoM to obtain more audio and to see if our note segmentation performs better this time. Our new audio interface and XLR cable also arrived this week, so we will hopefully be able to collect better audio samples as well. In the upcoming week, we will focus on:
- Polishing our STE/note segmentation
- Fixing the issues with making our sheet music editable via the Flat API
- Collecting user metrics such as their transcription history
- Deploying our web app
- Preparing our final presentation/demo
- Thorough testing of our pipeline
Below is our plan for verification, which we already started last week.
