Posted on

Carlos’s Status Report for 3/6/2021

This week, I conducted considerable research into features a singer’s vocal performance that can be used to discriminate between good and bad singing. I stumbled upon a few papers that discussed how we can do this, most importantly this one and this one.

In the first paper, the authors described 12 desirable characteristics used to define good singing, as described by experts in the field, and they adapted existing methods that measure those traits and aggregated them to generate a metric which they call Perceptual Evaluation of Singing Quality (PESnQ). However, to obtain this measure, a singer’s performance must be compared to an exemplary performance, which is out of the scope of our project. Most literature in this field of singer evaluation follows this methodology of using a template by which to compare a performance to. Continue reading “Carlos’s Status Report for 3/6/2021”

Posted on

Team Status Report for 3/6/2021

This week, we continued to flesh out our project idea and research our respective components of the project. We have not had any major changes since, but we have considerably developed our ideas in preparation of the design review. This new schedule reflects the changes made to our project from last week and our new deadlines. We grouped our planning into three phases, the end of each stage signifying the start of integration between disjoint parts of the project.

The biggest risk that we face going forward is underestimating the time it will take to complete our respective tasks and integrate our parts into a system. To manage this risk, we have built-in slack time to our deadlines and given ourselves hopefully ample time to integrate. Also, as with any big project, we will be setting up version control via GitHub to mitigate issues where local changes introduce integration bugs and disrupt the project as a whole.

We also attached a wireframe of what the user will experience in using our app to guide us in ideation and in thinking about the corner cases and integration issues that we may have overlooked otherwise.

 

Posted on

Sai’s Status Report for 3/6/21

This week, I worked on:

  • Flow diagram for optical flow of the web application 
  • System Diagram of User inputs + Application Outputs
  • Feedback Visuals
  • Explicit Wireframes for Pitch Exercise (Scale) and Rhythm Exercise (Clapping) Pages
  •  
  • Voice Range Evaluation will be a part of the user registration, came up with visual representation for that
  • Worked on design review slides:  https://docs.google.com/presentation/d/1T3X5lNBo8QEb4_bPB_0_1S9zrcxBrfxpWwo3TOec-fM/edit?usp=sharing
  • Came up with testing for the User Experience – Focus Group surveys
    • Will come up with a few tasks for users to work on (navigating to a specific exercise/page, doing an exercise)
    • Will have them screen share their activity on the web application and take note of how long it takes for them and their ease of use

This upcoming week:

  • Work on design report
  • Start coding the UI for one of the pitch exercise pages (work on html template)
  • Start Django code on UI for rhythm exercise page (html template)

 

 

 

Posted on

Sai’s Status Report for 2/27/21

This week, I presented our proposal and received some feedback from Professor Sullivan on simplifying our product more. We acknowledged that providing real-time feedback on audio input and video input will have many latency issues. This has pushed us to simplify our project.  Since our team shifted the product’s use case from users being able to practice songs to doing short singing exercises, without real-time feedback, but with feedback after recording, I was able to brainstorm a few ideas for singing exercises and how the user interaction would roughly look like for them.  I attached the wireframes for those below.

I was also able to design an UX flow/system integration diagram for what our team planned as our new design for the product’s transition to non-realtime and singing exercise focus. This diagram can be found in the team status report.

I also did some research on microphones with good noise cancelling ability and webcams with high resolution. We also decided today that we want the user to able to have headphones to listen to their audio. Here are some of the best options we have:

Camera: https://www.logitech.com/en-us/products/webcams/c270-hd-webcam.960-000694.html

Microphones/Headphones: https://www.jabra.com/business/office-headsets/jabra-evolve/jabra-evolve-75##7599-832-109

My progress is a little behind of what I planned, but this is due to the pivot in our design that we planned originally before our proposal presentation. Some actions I will take to catch up to the project schedule will be to put in some research and work into the project for at least an hour almost everyday this week with documentation/notes and to plan according to the design presentation specifications.

For next week, I want to think of other singing exercises we could implement and how the UI would look for those. I want to get a comprehensive understanding of how WebAudio API works and plan out exactly how I will use it for the UI. I also want to figure out the best way (best visualizations/infographics) to show users their feedback on their singing and their progress reports for their overall performance history. I also want to plan out our Django Framework organization (views/models/controller) so that I can get started on coding that the next week.

Posted on

Carlos’s Status Report for 2/20/21

As discussed in our team status report, we have made many changes to the scope and goals of our project based on the feedback we received after presenting our proposal. Most notably, we will no longer be detecting pitch or rhythm in real-time, nor will we be evaluating a singer’s performance with respect to that of an uploaded song, both of which were aspects of the project that I was responsible for. We will not be implementing pitch detection in real-time because of unrealistic latency bounds. Now, pitch and rhythm detection and feedback will be provided after a performance. This makes pitch detection significantly easier because there already exist several well-researched pitch detection algorithms (PDAs). I will be implementing our pitch detector using the autocorrelation method, which excels in estimating monophonic pitch. I plan on implementing this pitch detector by the end of this week.

Given that our app will no longer provide real-time feedback, we decided that it would be nice to include more features that are indicators of good singing. One such feature is the phonetogram which measures a singer’s singing intensity at a given frequency, thus a good indicator of a singer’s range.

I have also very recently come across a wholistic singing quality metric called the Perceptual Evaluation of Singing Quality (PESnQ) score as described here. I see great promise in this metric for our purposes and will read the paper in more detail. With this metric, I think we have enough to provide users’ with sufficient feedback on their performance.

Posted on

Funmbi’s Status Report: 2/27/2021

So this week we had to finalize the information for our proposal presentation, which we all worked on collectively. My main focus was on posture analysis, so I worked on what technical challenges we would face when it comes to posture regarding good singing posture. However, after our presentation, we were faced with the complexity of real-time analysis which when coupled with latency becomes more difficult to handle within the time frame that we currently have for our project.
Therefore we had to pivot our analysis and focus on a lesson-based composition. Because of that, I spent most of this week looking into musical exercises and the music theory that we consider important to start out music lessons for beginners. I worked on narrowing down the scale for lessons to just the major scale to reduce the complexity we would have to work on with other scales (minor -> natural, harmonic, etc)

Continue reading “Funmbi’s Status Report: 2/27/2021”

Posted on

Team Status Report for 2/27/2021

One major risk that we can potentially face at this point of the project is having diverging ideas of our project due to working on our individual components without integration in mind, which can cause us problems down the road. A problem we are actively solving now is clearly defining our project scope and goals. After our proposal presentation, we pivoted from a real-time feedback singing coach to one that provides feedback after a performance. This change was brought on by the difficulty of balancing pitch analysis and reasonable latency constraints to allow for real-time feedback. Our project now focuses on providing lessons in different singing aspects, namely: pitch accuracy, timing, and music theory. We have also made song performance analysis a stretch goal for our project, which used to be a core feature.

These changes in project goals and scope have prompted us to re-evaluate our project schedule, something that we are still updating as we continue to enumerate the aspects of our updated project. We have been meeting more than usual this week (5 instead of 4 times, for 1-2 hours) to address the changes in our project. We documented our pivot ideas in this document here:

https://docs.google.com/document/d/1hAuto5vd_O6j6cV3cd6imi2Ym6Bqz51UVvyWXM9Fn9g/edit?usp=sharing

As we are now taking a more lesson based approach, we are in the process of crafting lessons to help train users in the aspects of singing mentioned above. We are considering using  a phonetograph to analyze users’ vocal ranges upon signup and compute it after good performance in several lessons to measure progress over time. After signup, they will go through a range of pitch exercises and rhythm exercises. Posture analysis is also no longer being provided in real time, but after the lessons as a part of the performance report.

We designed the integration and flow of this new lesson based approach as a rough draft which can be found below:

 

Posted on

Team Status Report for 2/20/21

One of the main risks our team might be facing is not having a clear definition of baseline features and functionalities of product. Another risk is underestimating the time requirement and difficulty of each task. These two risks can be managed by further researching how feasible each possible feature could be and narrow them down to a smaller set of features to implement. We are in the early stages of our design and just came up with our first prototype.

 

Posted on

Sai’s Status Report for 2/20/21

This week, me and my team were able to narrow down and finalize our baseline requirements for the project based on feedback from Professor Sullivan. I came up with a prototype for the user interface of the web application and got started on a story board for the user experience with the app.  I feel that my progression is on schedule. I’ll be presenting our proposal so I need to prepare my presentation delivery soon. For the next week, I hope to be able to come up with a final User Interface and Storyboard design for the web application, begin research on Web Audio API, and get a start on the basic wireframe of the application.

 

Posted on

Carlos’s Status Report for 2/20/21

This week I’ve been researching additional vocal features for our system that work well as discriminators for good and bad singing. The best metric that I’ve seen so far is called the Singing Power Ratio (SPR). Next week, I will start implementing the real-time pitch and timing detection systems.