Posted on

Team Status Report 4/24/2021

These past few weeks our team has been working on each of us finalizing the last few functionalities for our project.  Some risks that we might run into is issues when we do integrate our separates part of the project, as we have delayed the integration back this might run into bugs. We plan on starting this coming week and would be meeting more frequently to best work on finalizing the project. There is also the issue of having it all ready to get feedback from other people so we can edit it in time for the presentation and hopefully make the necessary changes that they might highlight.

We included the addition of pauses between each note for pitch detection to allow for easy analysis of the pitch, and this is because without those breaks there is ambiguity to where one note ends and another note starts because a user’s pitch could range up to 50 cents which is halfway between a semitone which can affect the analysis and the scoring we may give the user on their pitch accuracy, and in turn the effectiveness of our application. In addition we also added a countdown for the pitch exercises so the user knows when to sing and can also assist in analyzing to know where to start picking up the user voice for pitch detection.

Posted on

Carlos’s Status Report for 4/10/2021

I hoped by this week to have completed the pitch to tone mapping algorithm, but it is a much more involved endeavor than I anticipated. I have been having difficulties enumerating the cases that I have to consider for when to detect notes, most importantly determining when the user starts singing, if they keep their time alignment throughout the duration of the song, and how to handle the cases where they are not singing at all. For much of my time developing, I have treated this algorithm as if it were simply performing the pitch to tone mapping, but in reality there are several aspects of the users’ performance that I have had to consider. Most recently, I have found more success in taking a top-down approach and sectioning the responsibilities of the algorithm by function.

I am currently trying to finish this algorithm up, once and for all, by the end of this weekend, so that my team and I can integrate our respective parts and construct a preliminary working system. I am not sure if I will be able to test the algorithm as exhaustively as I should, so I will set a first round of unit tests on the generated pure tones.

Posted on

Sai’s Status Report for 4/10/21

This past week, I was able to figure out how to send the audio files that a user records for pitch exercises back to the server and store it into the database at the same time through AJAX. I also realized that for generating piano notes for the user to listen to and imitate, web audio api was not the best solution as it produced very robotic sounds for different notes. I found another library called AudioSynth which is very easy to use and generated very realistic piano notes. However, I ran into a roadblock where I failed to consider how the lessons would be generated. We planned that the lesson exercises for scale exercises in particular would be sent to the server and processed in javascript as dictionary with the pitch notes (A, B, C, D, etc) as keys and their durations (eighth note, quarter note, half note, etc) as their value in the order that it would be played and required to be sung. This will require JSON format in order to store them and send to the server for the pitch detection algorithm and feedback algorithm to compare the user’s pitch to the expected pitch. However this would mean that these scale exercises would need to be stored and hardcoded in javascript and sent in JSON file format to the server through AJAX, but there needs to be a more efficient way to storing/generating these pitch/scale exercises, which is what I need to figure out this upcoming week.

I’ve made a significant amount of progress this past week, however, I feel that I’m behind on progress in regards to integrating my code with the server. Therefore, figuring out how to get over the roadblock will require some planning which I plan to do this week after I generate feedback from the exercise to present for the interim demo. But before planning the integration, I plan on just using hardcoded values for the lessons without integration for now given the time constraints.

Here’s the code I wrote to send audio recordings over to the server via AJAX:

Here’s the code I used to generate piano notes for a C Major Scale:

 

Posted on

Funmbi’s Status Report: 4/10/2021

This week was rather difficult, I hit a problem with the javascript and python file integration. I had been hardcoding most of the exercises so I just finally started integrating it with Django. So I spent most of the week trying to fix that . I came up with a new solution that would allow me to work with AJAX. A screenshot of what I am testing so far to store the feedback for the exercises is below:

Continue reading “Funmbi’s Status Report: 4/10/2021”

Posted on

Team Status Report for 4/10/21

This past week, we’ve noticed a few significant risks to the progress of our project. The first risk is that we won’t be well-prepared to integrate our front-end code with the server. As we’ve been spending most of our time hard-coding the exercises to make sure it looks right on the front end, we have run into a couple of road blocks as we haven’t planned out exactly how to send back expected exercise results to the server to compare it to the user’s actual results/recording and if it will work. The second risk, is that we won’t have enough time to be able to get our MVP.  We have to account for the time that we need spend on other classes as well since they’ve been taking up more and more of our time these past few weeks.

To mitigate these risks, we will plan out and collaborate more to plan and lay out all the functions we will be using in the different components of the web application – the views.py file( on the server ),  html + javascript ( frontend) and how they will be linked together.  As for accounting for the time restrictions we all have on doing work for the capstone project, we will all try to plan out how many hours we can spend on the capstone at the beginning of each week and make sure it’s at least 12, without other coursework taking too much time and without too much time spent on capstone.

There have been a few design changes made to our project. One of the design changes is that for the music theory exercises, instead of using VexFlow, we will be using image files to present the lessons due to the integration roadblocks mentioned above. Another design change we made was that instead of using Web Audio API to generate piano notes, we will be using another library we found online: AudioSynth since the AudioSynth library has more code to play more natural piano tones, whereas web audio api generated very robotic tones.

Our schedule has changed slightly:

Here’s a picture of our progress on the frontend of a basic pitch exercise:

Posted on

Carlos’s Status Report for 4/3/2021

This week, I continued developing the pitch to key mapping that takes in a pitch contour as generated by the Praat pitch detector, and outputs a set of time stamps and tones corresponding to pitch and rhythm. This component turned out to be more intricate than I had initially expected due to confusion arising from a lack of formal music understanding. Particularly, I was having trouble understanding the relationship between tempo, time signatures, and note duration. Continue reading “Carlos’s Status Report for 4/3/2021”

Posted on

Sai’s Status Report for 4/3/2021

This week, I started thinking about how to MediaRecorder API with our django web app, and the best way to send back the audio recording to the django server and how to store it. I was able to use the MediaRecorder functionality in the javascript file of our html + css + js frontend and got the basic recording and playback feature implemented:

I have also set up the data model to store the recording for a user for a given lesson in the Lesson Model:

 I have planned to link this model to a form that will send the audio recording back to the server as a POST request. This still needs needs to be implemented in code and tested. That’s one of the tasks I plan to complete earlier in this week.

I am still behind schedule, as I planned to be working on setting up the feedback page for the user’s audio recordings but hopefully, if I can manage to finish being able to record a user’s audio and store it, and pass it back to our pitch detection algorithm in the backend, I will immediately start working on the feedback generation and rendering at the end of this week and hopefully get that done before the interim demo.

 

Posted on

Funmbi’s post: 4/3/2021

This week I worked on finishing the theory for the user to read to practice the music exercises that I created last week. I had to study a lot of different sites to compile the theory and as well as consider the information that would be important to a beginner. I also worked on fixing the java script compatibility issues when running our program on the Django Framework. So now I have the  exercises corresponding to the lessons on the base template html.

Continue reading “Funmbi’s post: 4/3/2021”

Posted on

Sai’s Status Report for 3/27/2021

This past week, I was able to get started on coding the web application’s HTML templates and integrated bootstrap features with it. I was able to finish getting the login, registration, and user dashboard pages implemented.  I’ve spent too much time focusing on the visual aspect (integrating bootstrap) of the web application, trying to make each element look perfect, and I realized that this has been making me fall behind schedule. I will be trying to implement the basic skeleton of the web application’s essential and main features first before getting bogged down on the visual details. Minor progress item, but I also placed the order for the headset we plan on using.

Pictures of the login and register pages are uploaded in the team status report. Here’s a picture of the user dashboard page:

For the upcoming week, I hope to finish getting a basic and simple pitch exercise page and a basic rhythm exercise page implemented as well as get started on the actions that control the functionality of the django application. I will also be working on implementing the database models that will store user info.

Posted on

Funmbi’s Status Report: 3/27/2021

This past 2 weeks I spent understanding the APIs I would be working with and then incorporating that into the current code. So I worked mainly with VexFlow because I spent this week working on the music theory lessons. I built the finding notes on a scale exercise such that the user would get a random lesson they could try every time. I also worked on the feedback for that lesson so it was understandable to the user at first glance. With this code I can all generate the clef exercises as well as the duration exercises using the same format, so I aim to finish that by Wednesday next week.

I also started working on the resources page and gathering external resources the user would find helpful to add to the page. I think that page would be a working progress as new resources can be collated at any time, but I have the html for that page started and so that is not a worry.

Continue reading “Funmbi’s Status Report: 3/27/2021”