Posted on

Sai’s Status Report for 2/27/21

This week, I presented our proposal and received some feedback from Professor Sullivan on simplifying our product more. We acknowledged that providing real-time feedback on audio input and video input will have many latency issues. This has pushed us to simplify our project.  Since our team shifted the product’s use case from users being able to practice songs to doing short singing exercises, without real-time feedback, but with feedback after recording, I was able to brainstorm a few ideas for singing exercises and how the user interaction would roughly look like for them.  I attached the wireframes for those below.

I was also able to design an UX flow/system integration diagram for what our team planned as our new design for the product’s transition to non-realtime and singing exercise focus. This diagram can be found in the team status report.

I also did some research on microphones with good noise cancelling ability and webcams with high resolution. We also decided today that we want the user to able to have headphones to listen to their audio. Here are some of the best options we have:

Camera: https://www.logitech.com/en-us/products/webcams/c270-hd-webcam.960-000694.html

Microphones/Headphones: https://www.jabra.com/business/office-headsets/jabra-evolve/jabra-evolve-75##7599-832-109

My progress is a little behind of what I planned, but this is due to the pivot in our design that we planned originally before our proposal presentation. Some actions I will take to catch up to the project schedule will be to put in some research and work into the project for at least an hour almost everyday this week with documentation/notes and to plan according to the design presentation specifications.

For next week, I want to think of other singing exercises we could implement and how the UI would look for those. I want to get a comprehensive understanding of how WebAudio API works and plan out exactly how I will use it for the UI. I also want to figure out the best way (best visualizations/infographics) to show users their feedback on their singing and their progress reports for their overall performance history. I also want to plan out our Django Framework organization (views/models/controller) so that I can get started on coding that the next week.

Posted on

Carlos’s Status Report for 2/20/21

As discussed in our team status report, we have made many changes to the scope and goals of our project based on the feedback we received after presenting our proposal. Most notably, we will no longer be detecting pitch or rhythm in real-time, nor will we be evaluating a singer’s performance with respect to that of an uploaded song, both of which were aspects of the project that I was responsible for. We will not be implementing pitch detection in real-time because of unrealistic latency bounds. Now, pitch and rhythm detection and feedback will be provided after a performance. This makes pitch detection significantly easier because there already exist several well-researched pitch detection algorithms (PDAs). I will be implementing our pitch detector using the autocorrelation method, which excels in estimating monophonic pitch. I plan on implementing this pitch detector by the end of this week.

Given that our app will no longer provide real-time feedback, we decided that it would be nice to include more features that are indicators of good singing. One such feature is the phonetogram which measures a singer’s singing intensity at a given frequency, thus a good indicator of a singer’s range.

I have also very recently come across a wholistic singing quality metric called the Perceptual Evaluation of Singing Quality (PESnQ) score as described here. I see great promise in this metric for our purposes and will read the paper in more detail. With this metric, I think we have enough to provide users’ with sufficient feedback on their performance.

Posted on

Funmbi’s Status Report: 2/27/2021

So this week we had to finalize the information for our proposal presentation, which we all worked on collectively. My main focus was on posture analysis, so I worked on what technical challenges we would face when it comes to posture regarding good singing posture. However, after our presentation, we were faced with the complexity of real-time analysis which when coupled with latency becomes more difficult to handle within the time frame that we currently have for our project.
Therefore we had to pivot our analysis and focus on a lesson-based composition. Because of that, I spent most of this week looking into musical exercises and the music theory that we consider important to start out music lessons for beginners. I worked on narrowing down the scale for lessons to just the major scale to reduce the complexity we would have to work on with other scales (minor -> natural, harmonic, etc)

Continue reading “Funmbi’s Status Report: 2/27/2021”

Posted on

Team Status Report for 2/27/2021

One major risk that we can potentially face at this point of the project is having diverging ideas of our project due to working on our individual components without integration in mind, which can cause us problems down the road. A problem we are actively solving now is clearly defining our project scope and goals. After our proposal presentation, we pivoted from a real-time feedback singing coach to one that provides feedback after a performance. This change was brought on by the difficulty of balancing pitch analysis and reasonable latency constraints to allow for real-time feedback. Our project now focuses on providing lessons in different singing aspects, namely: pitch accuracy, timing, and music theory. We have also made song performance analysis a stretch goal for our project, which used to be a core feature.

These changes in project goals and scope have prompted us to re-evaluate our project schedule, something that we are still updating as we continue to enumerate the aspects of our updated project. We have been meeting more than usual this week (5 instead of 4 times, for 1-2 hours) to address the changes in our project. We documented our pivot ideas in this document here:

https://docs.google.com/document/d/1hAuto5vd_O6j6cV3cd6imi2Ym6Bqz51UVvyWXM9Fn9g/edit?usp=sharing

As we are now taking a more lesson based approach, we are in the process of crafting lessons to help train users in the aspects of singing mentioned above. We are considering using  a phonetograph to analyze users’ vocal ranges upon signup and compute it after good performance in several lessons to measure progress over time. After signup, they will go through a range of pitch exercises and rhythm exercises. Posture analysis is also no longer being provided in real time, but after the lessons as a part of the performance report.

We designed the integration and flow of this new lesson based approach as a rough draft which can be found below:

 

Posted on

Team Status Report for 2/20/21

One of the main risks our team might be facing is not having a clear definition of baseline features and functionalities of product. Another risk is underestimating the time requirement and difficulty of each task. These two risks can be managed by further researching how feasible each possible feature could be and narrow them down to a smaller set of features to implement. We are in the early stages of our design and just came up with our first prototype.

 

Posted on

Sai’s Status Report for 2/20/21

This week, me and my team were able to narrow down and finalize our baseline requirements for the project based on feedback from Professor Sullivan. I came up with a prototype for the user interface of the web application and got started on a story board for the user experience with the app.  I feel that my progression is on schedule. I’ll be presenting our proposal so I need to prepare my presentation delivery soon. For the next week, I hope to be able to come up with a final User Interface and Storyboard design for the web application, begin research on Web Audio API, and get a start on the basic wireframe of the application.

 

Posted on

Carlos’s Status Report for 2/20/21

This week I’ve been researching additional vocal features for our system that work well as discriminators for good and bad singing. The best metric that I’ve seen so far is called the Singing Power Ratio (SPR). Next week, I will start implementing the real-time pitch and timing detection systems.

Posted on

Funmbi’s Status Report : 2/20/2021

So far we have been working on fine tuning our product, and inspecting the metrics we would like our software to cater to for the user. We have completed the abstract and finished the slides for our proposal. After the last 2 meetings with a professor and TA we have a baseline for our project and are currently researching the most effective approach. The work has been divided based on our skills and I am currently working on how to use the open pose library for the posture detection. This week I am going to be collating data sets for the posture detection for testing and thinking of ways to detect posture with the microphone in the way, which was an issue raised by my group.

In addition I also need to think on how to integrate the database to the web application as well as finding songs that would be our base exercises for the users as well as test cases. Basically a lot of this week and the upcoming week is on research, and finding the software that would be preferable for the product and how to integrate it to fit the user experience.