Team Status Report for 2/18

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Right now, an aspect that is somewhat concerning to us is the accuracy and confidence of the pitch detection modules we are looking at. From tests that we have done, the aubio pitch tracker sometimes does not have high confidence in its pitch detection. It performs better when the vocals are loud and are simple, vowel sounds. However, we are still just beginning to work with it, so we expect to make some progress. Nonetheless, we have two plans that we hope will improve this performance.

1. We have ordered and received our hardware, a headset microphone and interface. We haven’t had much time to experiment with these, but we expect that using the mic will reduce background noise and increase audio quality, so this should help the performance of aubio.
2. While we have been working with aubio mostly, we are also planning to experiment with and try making progress on other pitch tracking modules. This week and next are both dedicated to testing different modules, so we plan to perform more experiments while we find one that performs with good confidence and accuracy.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?  

After discussing the difficulty and scope of the project, we have decided to use an existing pitch recognition module. Because latency in pitch recognition is so important to us, using these existing modules, whose backend implementation is written in C, would make fast pitch recognition much easier to achieve. We expect this is easier, but it also means we will have less control over the pitch detection. Kelly is currently working on testing out different pitch detection algorithms, in order to understand which performs better, which is most compatible with our use case. Additionally, we have ordered and received our hardware, which will allow us to test the algorithms in the environment they will be used in our final project.

Our hardware for this project has gone through several iterations, especially this week. Originally, we were planning to have a fairly cheap headset microphone, but after consulting with our TA, Ankita, and our Professor, Tom Sullivan, we decided to revise our plans. Professor Sullivan talked a lot about minimizing the outside noise as much as possible, specifically for testing our designs and so we quickly pivoted to a headset microphone as this ensures that the microphone will be close to the users’ mouth and minimizing background noise. After meeting again, we started looking into a wireless audio system that live performers would use, but Professor Sullivan warned against wireless as this could distort our signal thereby throwing off our pitch tracking algorithm. We finally landed on a headset microphone primarily used by professional streamers/broadcasters as well as an interface in order to ensure that our computers would be able to read the signal coming from our high quality microphone. This change was necessary, especially for testing our pitch tracking algorithms as it will ensure that the signal coming in is much cleaner than that of our builtin laptop microphone and gives the pitch tracking algorithms the best chance at performance. However, this does come at the cost of ~$330. As this is the only part we planned on spending money on for this project, this cost has not affected our overall project.

Additionally, we are currently switching focus on having a series of hardcoded melodies and songs, while we originally were considering allowing users to upload their own. Although this would be an interesting feature, we don’t think it is really necessary to achieve the goal of a prototype for a responsive pitch detecting karaoke game. This decreases the scope of our project, both because we don’t need to worry about processing inputted songs, but also because we can have more control over what kind of songs will be tested on. Currently, we have a list of both simple melodies and songs that we want to implement for our final project.

A benefit of allowing users to upload their own songs is that users would be more engaged with the game and generally have a more enjoyable experience. To mitigate this, we tried to choose songs and melodies that are well known, somewhat easier, to appeal to beginners, and enjoyable to sing. The fact that these songs are popular will also make it easier for us to determine the target pitch for the user to hit, as many of them may have this information available. Additionally, sticking to hard coded songs allows us to stick to songs with slower and more defined melodies, which we suspect will perform better with our pitch detection.

Based on both of these changes, we have had to change the distribution of work, as well as the gantt chart. We are no longer spending time working on a homegrown algorithm, with a new focus on exploring existing algorithms. This has taken some of Anita’s tasks off the table, so she is now taking on some of the web development tasks. We have also spent some time fleshing out and separating the tasks for web development, now that we have had a bit more time to discuss exactly what we want to see on our site and how we want to organize it.

This is also the place to put some photos of your progress or to brag about a component you got working.

Testing of pyaubio. Code on the left, pitch detection and accuracy on the right.

Integration of fonts into the website framework.

The game homescreen now contains links to two hardcoded songs, twinkle twinkle little star and happy birthday. For both of these, this takes us to a page that plays this song. Mp3, wav, and ogg formats for audio files are currently supported.

Please enumerate one or more principles of engineering, science and mathematics that your team used to develop the the design solution for your project.

Because we are still in the beginning stage of the project, the largest principle we are working on is engineering management. This has consisted of adjusting the scope of the project as we go, and, in doing so, keeping solid communication, and tweaking and adjusting our personal responsibilities and plans when it comes to the scope of the project. Anita’s tasks have changed this week, and Kelly has taken on some new responsibilities, which has resulted in Anita taking on web development tasks later in the semester while she focuses on the design presentation this week.

Our project includes considerations for __________, where the blank lists things like public health, safety and welfare, along with other global, cultural, societal, environmental and economic considerations. Please add a second sentence to justify each item in the list in your blank

For this project, our largest consideration is still accessibility. We want to provide an enriching and enjoyable experience for those who like karaoke but may want to improve their skills. This is definitely something that we are taking into account when we choose the hardware we want to use. The headset mic and interface we are using come out to about $300, which is more than most at home karaoke machines on the market. However, we hope to provide a more valuable experience than a karaoke machine by being able to give feedback on user performance. Additionally, we are hoping that this equipment will not be necessary for every user. For development, we chose a pretty high quality microphone that had been recommended by streamers and podcasters, but we are expecting that a lower quality microphone will still be able to do the job in the finished product. Lastly, we expect that a non-negligible amount of users may already own a microphone and headphones, or would be able to use the microphone and headphones for other purposes in addition to KaraoKey, reducing the barrier of entry further.