Month: April 2021

This week, there were some file formatting issues for the recorded audio files that were submitted on the web application. I found out that the API I’m using to record a user’s audio input on a browser doesn’t produce uncorrupt/clean .wav files that you can open or process on the backend, but produces clean .webm  files. I found an ffmpeg package that I can call on the server to convert .webm files to .wav files. This package has been successful and produces clean .wav files. We made some design changes to our pitch exercises, there is now  a countdown when you press the record button and we wanted to add breaks in between notes when listening to how the scale/interval sounds via piano notes. I had to change the algorithm that produced the sequence of piano notes and displays/animates the solfege visuals. There is a now a dashboard/list of some of the different pitch scale/interval exercises. But now, new exercises can be easily added by just adding a model with its sequence of notes, durations, and solfege.  I also spent some time refining the solfege animation/display since some edge cases came up with adding more pitch exercises. I want to add a more refined display for the solfege display this upcoming week using some bootstrap features. Another minor edit I made was adding a nice logo for web app and making the title bar more engaging/bright.  I feel like I’m behind schedule, since I thought integration would happen this past week, but we will spend a lot more time on this project together to make sure the integration gets done before next week.

For the upcoming week, I’ll be working on creating the voice range evaluation that will be part of the registration process, integrating with Funmbi’s features, and working with her to create the web app’s main dashboard. I will also integrate my code with Carlos’ Pitch detection algorithm to receive feedback from the audio recordings. I will also try to plan out what the User Experience survey questions/prompts will look like.

Below are some pictures of the new exercise format and the pitch exercise dashboard.

This past week, I was able to figure out how to send the audio files that a user records for pitch exercises back to the server and store it into the database at the same time through AJAX. I also realized that for generating piano notes for the user to listen to and imitate, web audio api was not the best solution as it produced very robotic sounds for different notes. I found another library called AudioSynth which is very easy to use and generated very realistic piano notes. However, I ran into a roadblock where I failed to consider how the lessons would be generated. We planned that the lesson exercises for scale exercises in particular would be sent to the server and processed in javascript as dictionary with the pitch notes (A, B, C, D, etc) as keys and their durations (eighth note, quarter note, half note, etc) as their value in the order that it would be played and required to be sung. This will require JSON format in order to store them and send to the server for the pitch detection algorithm and feedback algorithm to compare the user’s pitch to the expected pitch. However this would mean that these scale exercises would need to be stored and hardcoded in javascript and sent in JSON file format to the server through AJAX, but there needs to be a more efficient way to storing/generating these pitch/scale exercises, which is what I need to figure out this upcoming week.

I’ve made a significant amount of progress this past week, however, I feel that I’m behind on progress in regards to integrating my code with the server. Therefore, figuring out how to get over the roadblock will require some planning which I plan to do this week after I generate feedback from the exercise to present for the interim demo. But before planning the integration, I plan on just using hardcoded values for the lessons without integration for now given the time constraints.

Here’s the code I wrote to send audio recordings over to the server via AJAX:

Here’s the code I used to generate piano notes for a C Major Scale: