Category: Team Status Reports

This past week, our team was able to successfully integrate the user interface with the pitch detection and clap detection algorithms and the web application now displays and stores feedback for pitch and rhythm to a user’s audio recording.  Therefore, we no longer have any potential risks in integration. However before this upcoming Monday, the only risk we face is not being able to deploy our web application with the EC2 Instance. We tried to deploy more than once on the EC2 Instance using amazon S3 for storage, however, the web application is unresponsive at the public IP address on HTTP/HTTPS ports. We are currently still trying to make this deployment work, but our risk mitigation is to deploy on an 8000 port since that seems to be loading the web application on the EC2 instance. However we need to also use SSL for this option  in order to get the audio recording functionality working. Furthermore, another risk mitigation, will be to use SQL for database storage instead S3 since using S3 with the EC2 Instance is not as straightforward to deploy as is SQL. This will mean that there could potentially be design change for our product. No other parts of the design will be affected, the database storage will be the only thing that is replaced. Until Monday, we will be attempting to get deployment to work, working on the poster, the demo video, and making some User Interface refinements. Our schedule remains the same. Below are some pictures of the integrated feedback for the exercises.

These past few weeks our team has been working on each of us finalizing the last few functionalities for our project.  Some risks that we might run into is issues when we do integrate our separates part of the project, as we have delayed the integration back this might run into bugs. We plan on starting this coming week and would be meeting more frequently to best work on finalizing the project. There is also the issue of having it all ready to get feedback from other people so we can edit it in time for the presentation and hopefully make the necessary changes that they might highlight.

We included the addition of pauses between each note for pitch detection to allow for easy analysis of the pitch, and this is because without those breaks there is ambiguity to where one note ends and another note starts because a user’s pitch could range up to 50 cents which is halfway between a semitone which can affect the analysis and the scoring we may give the user on their pitch accuracy, and in turn the effectiveness of our application. In addition we also added a countdown for the pitch exercises so the user knows when to sing and can also assist in analyzing to know where to start picking up the user voice for pitch detection.

This past week, we’ve noticed a few significant risks to the progress of our project. The first risk is that we won’t be well-prepared to integrate our front-end code with the server. As we’ve been spending most of our time hard-coding the exercises to make sure it looks right on the front end, we have run into a couple of road blocks as we haven’t planned out exactly how to send back expected exercise results to the server to compare it to the user’s actual results/recording and if it will work. The second risk, is that we won’t have enough time to be able to get our MVP.  We have to account for the time that we need spend on other classes as well since they’ve been taking up more and more of our time these past few weeks.

To mitigate these risks, we will plan out and collaborate more to plan and lay out all the functions we will be using in the different components of the web application – the views.py file( on the server ),  html + javascript ( frontend) and how they will be linked together.  As for accounting for the time restrictions we all have on doing work for the capstone project, we will all try to plan out how many hours we can spend on the capstone at the beginning of each week and make sure it’s at least 12, without other coursework taking too much time and without too much time spent on capstone.

There have been a few design changes made to our project. One of the design changes is that for the music theory exercises, instead of using VexFlow, we will be using image files to present the lessons due to the integration roadblocks mentioned above. Another design change we made was that instead of using Web Audio API to generate piano notes, we will be using another library we found online: AudioSynth since the AudioSynth library has more code to play more natural piano tones, whereas web audio api generated very robotic tones.

Our schedule has changed slightly:

Here’s a picture of our progress on the frontend of a basic pitch exercise:

This past 2 weeks, we started working on the bulk of the project. The headphones that we would be testing our website on was ordered and has arrived so we can do some testing before the interim demo. Some risks that we have run into and are likely.

While testing our project, in an attempt to run it using the Django Framework we realized Django was not recognizing one of our APIs and there might be an extra step to allow for external APIs to run on Django that we were not aware of, which took time from coding and slowed us down a little, but we would be meeting with a Web Development Professor at Carnegie Mellon with hopes to get it all ironed out before next week. Also while working we are also realizing the risk of time management. Spending time on the little bugs affecting the time we could spend on ironing out the bone structure without going into the complexities of our app. To handle that, we are currently working to make the basic app work on small tasks and hope to build on from there once we have the basic tasks working correctly.

We made one major change to our system spec. While testing the third party implementation of the Yin PDAwe realized the programmer did not implement all the steps required and so we discovered it did not meet the level of accuracy we expected to pass our requirements. In addition the code was also difficult to comprehend. For these reasons we have decided to use the Praat tool which runs on a python interface called Parselmouth. based on research we discovered that the Praat tool is better than the Yin algorithm, and we have begun working with this tool to implement our pitch detection for our web application.

Based on the changes and what we have implemented so far below is a screenshot of our updated schedule. So far, although slight changes had to be made there is still good hope that we are on track to having a viable product for demo.

Login Page:

Registration Page:

We finished our design presentation and began working on our design report this past week2. We finalized the final design of our product in terms of what will look like as well as what algorithms and features we will be implementing/creating vs. what algorithms and features we will be downloading or using open sources for.

The biggest risk we are facing right now is falling behind the schedule that we have set for ourselves. We have been focusing on making sure our web application is well-planned out to satisfy the requirements of the design presentation and design report and that’s where most of our time has been going, to the presentations and report. We haven’t budgeted time to focus on actually starting our work. 

We will try to mitigate this risk by trying to get our design report finished ahead of time so that we can get a start on implementing our individual tasks as soon as possible. As is it is midterm season for this past week and the upcoming week,  we definitely to need to try to proactively set time for implementation for a few hours during most of the days of the week.

No changes were made to our design, since we’ve received mostly positive feedback. However for the design report, we plan on organizing the main system diagram and color coding it based on off-the-shelf, own implementation, etc. We also need to consider the processing time + runtime to account for latency issues and mention this in our report.

For the upcoming week, we plan on working on our design report and beginning work on coding the UI, algorithms, and database implementation.

One major risk that we can potentially face at this point of the project is having diverging ideas of our project due to working on our individual components without integration in mind, which can cause us problems down the road. A problem we are actively solving now is clearly defining our project scope and goals. After our proposal presentation, we pivoted from a real-time feedback singing coach to one that provides feedback after a performance. This change was brought on by the difficulty of balancing pitch analysis and reasonable latency constraints to allow for real-time feedback. Our project now focuses on providing lessons in different singing aspects, namely: pitch accuracy, timing, and music theory. We have also made song performance analysis a stretch goal for our project, which used to be a core feature.

These changes in project goals and scope have prompted us to re-evaluate our project schedule, something that we are still updating as we continue to enumerate the aspects of our updated project. We have been meeting more than usual this week (5 instead of 4 times, for 1-2 hours) to address the changes in our project. We documented our pivot ideas in this document here:

https://docs.google.com/document/d/1hAuto5vd_O6j6cV3cd6imi2Ym6Bqz51UVvyWXM9Fn9g/edit?usp=sharing

As we are now taking a more lesson based approach, we are in the process of crafting lessons to help train users in the aspects of singing mentioned above. We are considering using  a phonetograph to analyze users’ vocal ranges upon signup and compute it after good performance in several lessons to measure progress over time. After signup, they will go through a range of pitch exercises and rhythm exercises. Posture analysis is also no longer being provided in real time, but after the lessons as a part of the performance report.

We designed the integration and flow of this new lesson based approach as a rough draft which can be found below:

 

One of the main risks our team might be facing is not having a clear definition of baseline features and functionalities of product. Another risk is underestimating the time requirement and difficulty of each task. These two risks can be managed by further researching how feasible each possible feature could be and narrow them down to a smaller set of features to implement. We are in the early stages of our design and just came up with our first prototype.