This week I worked on finalizing the feedback page and making it work. My aim was to get the feedback pages working and to get them displaying the correct data. As well as start fixing up the css of the pages for the final display to fit what we had desired for the end. Here are some pictures of my progress so far still have a few edits that need to be done but most likely would be done by the end of the week.
Team Status Report 4/24/2021
These past few weeks our team has been working on each of us finalizing the last few functionalities for our project. Some risks that we might run into is issues when we do integrate our separates part of the project, as we have delayed the integration back this might run into bugs. We plan on starting this coming week and would be meeting more frequently to best work on finalizing the project. There is also the issue of having it all ready to get feedback from other people so we can edit it in time for the presentation and hopefully make the necessary changes that they might highlight.
We included the addition of pauses between each note for pitch detection to allow for easy analysis of the pitch, and this is because without those breaks there is ambiguity to where one note ends and another note starts because a user’s pitch could range up to 50 cents which is halfway between a semitone which can affect the analysis and the scoring we may give the user on their pitch accuracy, and in turn the effectiveness of our application. In addition we also added a countdown for the pitch exercises so the user knows when to sing and can also assist in analyzing to know where to start picking up the user voice for pitch detection.
Carlos’s Status Report for 4/10/2021
I hoped by this week to have completed the pitch to tone mapping algorithm, but it is a much more involved endeavor than I anticipated. I have been having difficulties enumerating the cases that I have to consider for when to detect notes, most importantly determining when the user starts singing, if they keep their time alignment throughout the duration of the song, and how to handle the cases where they are not singing at all. For much of my time developing, I have treated this algorithm as if it were simply performing the pitch to tone mapping, but in reality there are several aspects of the users’ performance that I have had to consider. Most recently, I have found more success in taking a top-down approach and sectioning the responsibilities of the algorithm by function.
I am currently trying to finish this algorithm up, once and for all, by the end of this weekend, so that my team and I can integrate our respective parts and construct a preliminary working system. I am not sure if I will be able to test the algorithm as exhaustively as I should, so I will set a first round of unit tests on the generated pure tones.
Sai’s Status Report for 4/10/21
This past week, I was able to figure out how to send the audio files that a user records for pitch exercises back to the server and store it into the database at the same time through AJAX. I also realized that for generating piano notes for the user to listen to and imitate, web audio api was not the best solution as it produced very robotic sounds for different notes. I found another library called AudioSynth which is very easy to use and generated very realistic piano notes. However, I ran into a roadblock where I failed to consider how the lessons would be generated. We planned that the lesson exercises for scale exercises in particular would be sent to the server and processed in javascript as dictionary with the pitch notes (A, B, C, D, etc) as keys and their durations (eighth note, quarter note, half note, etc) as their value in the order that it would be played and required to be sung. This will require JSON format in order to store them and send to the server for the pitch detection algorithm and feedback algorithm to compare the user’s pitch to the expected pitch. However this would mean that these scale exercises would need to be stored and hardcoded in javascript and sent in JSON file format to the server through AJAX, but there needs to be a more efficient way to storing/generating these pitch/scale exercises, which is what I need to figure out this upcoming week.
I’ve made a significant amount of progress this past week, however, I feel that I’m behind on progress in regards to integrating my code with the server. Therefore, figuring out how to get over the roadblock will require some planning which I plan to do this week after I generate feedback from the exercise to present for the interim demo. But before planning the integration, I plan on just using hardcoded values for the lessons without integration for now given the time constraints.
Here’s the code I wrote to send audio recordings over to the server via AJAX:
Here’s the code I used to generate piano notes for a C Major Scale:
Funmbi’s Status Report: 4/10/2021
This week was rather difficult, I hit a problem with the javascript and python file integration. I had been hardcoding most of the exercises so I just finally started integrating it with Django. So I spent most of the week trying to fix that . I came up with a new solution that would allow me to work with AJAX. A screenshot of what I am testing so far to store the feedback for the exercises is below:
Team Status Report for 4/10/21
This past week, we’ve noticed a few significant risks to the progress of our project. The first risk is that we won’t be well-prepared to integrate our front-end code with the server. As we’ve been spending most of our time hard-coding the exercises to make sure it looks right on the front end, we have run into a couple of road blocks as we haven’t planned out exactly how to send back expected exercise results to the server to compare it to the user’s actual results/recording and if it will work. The second risk, is that we won’t have enough time to be able to get our MVP. We have to account for the time that we need spend on other classes as well since they’ve been taking up more and more of our time these past few weeks.
To mitigate these risks, we will plan out and collaborate more to plan and lay out all the functions we will be using in the different components of the web application – the views.py file( on the server ), html + javascript ( frontend) and how they will be linked together. As for accounting for the time restrictions we all have on doing work for the capstone project, we will all try to plan out how many hours we can spend on the capstone at the beginning of each week and make sure it’s at least 12, without other coursework taking too much time and without too much time spent on capstone.
There have been a few design changes made to our project. One of the design changes is that for the music theory exercises, instead of using VexFlow, we will be using image files to present the lessons due to the integration roadblocks mentioned above. Another design change we made was that instead of using Web Audio API to generate piano notes, we will be using another library we found online: AudioSynth since the AudioSynth library has more code to play more natural piano tones, whereas web audio api generated very robotic tones.
Our schedule has changed slightly:
Here’s a picture of our progress on the frontend of a basic pitch exercise:
Carlos’s Status Report for 4/3/2021
This week, I continued developing the pitch to key mapping that takes in a pitch contour as generated by the Praat pitch detector, and outputs a set of time stamps and tones corresponding to pitch and rhythm. This component turned out to be more intricate than I had initially expected due to confusion arising from a lack of formal music understanding. Particularly, I was having trouble understanding the relationship between tempo, time signatures, and note duration. Continue reading “Carlos’s Status Report for 4/3/2021”
Team Status Report for 4/3/2021
This week, we continued developing several core components of our project, particularly: lesson creation, web template design, and tone detection from calculated pitch contours. Thus far, we are on schedule and our pacing is generally what we anticipated.
While most of our design has stayed the same, we have made changes to how we will be storing data. As we have iterated over our project design and ideas, the data that we require have changed and to reflect that we updated our data model scheme from
to
.
This new scheme will provide us with more flexibility and reusability in how we represent and store data across users and exercises. Continue reading “Team Status Report for 4/3/2021”
Sai’s Status Report for 4/3/2021
This week, I started thinking about how to MediaRecorder API with our django web app, and the best way to send back the audio recording to the django server and how to store it. I was able to use the MediaRecorder functionality in the javascript file of our html + css + js frontend and got the basic recording and playback feature implemented:
I have also set up the data model to store the recording for a user for a given lesson in the Lesson Model:
I have planned to link this model to a form that will send the audio recording back to the server as a POST request. This still needs needs to be implemented in code and tested. That’s one of the tasks I plan to complete earlier in this week.
I am still behind schedule, as I planned to be working on setting up the feedback page for the user’s audio recordings but hopefully, if I can manage to finish being able to record a user’s audio and store it, and pass it back to our pitch detection algorithm in the backend, I will immediately start working on the feedback generation and rendering at the end of this week and hopefully get that done before the interim demo.
Funmbi’s post: 4/3/2021
This week I worked on finishing the theory for the user to read to practice the music exercises that I created last week. I had to study a lot of different sites to compile the theory and as well as consider the information that would be important to a beginner. I also worked on fixing the java script compatibility issues when running our program on the Django Framework. So now I have the exercises corresponding to the lessons on the base template html.