Carnegie Mellon ECE Capstone, Spring 2023 Project by Anita Ma, Kelly Woicik, Anna Gerchanovsky
What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). This week, we worked on the ethics …
What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? As of now, the current biggest risk is still the pitch tracking algorithm, especially since we just pivoted it to …
What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
This week, I focused on the ethics assignment, pivoting the pitch detection algorithm to read a .wav file as input, and testing the new pitch tracking algorithm.
This week, we were given an ethics assignment and came together as a team to discuss a potential worst use case of our project and how to go about fixing it.
Additionally, I created a new pitch tracking algorithm that intakes a .wav file instead of registering input directly from a live microphone stream. As our team pivoted to using MediaStream on our web application to record audio rather than PyAudio, we needed a way to communicate between MediaStream (in js) and our pitch tracker in Aubio (in python). Luckily, MediaStream can output a .wav file and Aubio can take in a .wav file, so I made a new Aubio only demo that taken in a .wav file and outputs pitches in Hz. The new algorithm can be found on our github here.
I then tested this new algorithm by recording several .wav files on my ‘C Major Scale Fast’ and ‘C Major Scale Slow’ files. I made 3 .wav files per C Major scale version (6 .wav files total): only the piano notes, myself humming to the file, and myself singing note names (i.e. Do Re Mi Fa …) to the file. I then put these .wav files through the new pitch tracker and graphed my results.
We begin with the C Major Scale Slow results:
As seen, the humming did a remarkable job of resembling the straight piano input signal. However, as soon as consonants via note names were added, we had far more outlier peaks due to the excess expended air. This can also been in the confidence as the humming stays relatively the same confidence with few rapid changes while the note names confidence jumps all over the place. These results don’t surprise me and keep me relatively hopeful for singing actual words as the relative shape of the note names pitches with the consonant peaks taken out hits the correct pitch range of the piano.
We now look at the C Major Scale Fast results:
Once again, we have very similar results to the C Major Slow outputs. I will say that I expected the pitch tracker to not register the note changes fast enough, but was pleasantly surprised with the pitches of the signals that had been picked up.
Overall, these graphs seem to suggest that if we throw out the consonant outliers in the pitch graphs, we’ll be able to have a remarkably noiseless signal with our current microphone and interface setup in a relatively quiet environment.
The .txt outputs from all of these .wav files detailing pitch and confidence can be found on our github here along with the .wav files the outputs correspond to. Also a link to the full spreadsheet with datapoints and graphs can be found here.
Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
This week, I wanted to make my new pitch detection algorithm, test it, and ensure that it would be able to perform appropriately when given user input. I would say that all of this was accomplished this week. As we had to pivot our project to using MediaStream, we got a little backtracked with the pitch detection algorithm being attached to the web application frontend. However, now that we’ve flushed out the new algorithm, I’m hopeful that this integration will happen soon.
What deliverables do you hope to complete in the next week?
Next week I hope to integrate my new algorithm with Anna and/or Anita’s portions of the project.
What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). So after figuring out the high level …
What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready? There was a lot of feedback given from peers and the instructors on the design review presentation that highlighted potential …
What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
Much, if not all, of this week was spent on the design review report. Writing this report up took substantially longer than I expected. There was a lot of missing justification that needed sources and evidence, so a lot of this week was bookkeeping and research. It feels a bit odd to be justifying our design so late in the game, but it was very helpful fleshing out the minute details of this project. It was especially helpful for our testing plan 😊
The design review report can be found under “Design Review,” where I contributed to the Introduction, Use-Case Requirements, and Design Trade Studies.
Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
I am in the feedback generation phase in my gantt chart, so I am still on progress. I will need to pick up the pace though, especially as I took all of Spring Break off. To avoid veering off schedule, I will aim to have the feedback generation code done next week and aim to hook this system up to Kelly’s work the week after that.
What deliverables do you hope to complete in the next week?
Have code stub done for the feedback generation.
What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). I spent the majority of my hours …
What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). This week, our focus as a team …
What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?
As of now, our most significant risks come from our pitch detection algorithm being accurate as well as our web application latency being able to both send audio into a python module and display complicated graphics.
As Kelly is working on the pitch detection algorithm, she’s taking her time in testing it out in the following environments:
She’s also currently testing on pitch detection on the following:
With time and thought put into testing as well as considerations put into why the algorithm may be failing (i.e noisy environment vs. the note changes happen too fast), we’re hoping that we’ll have a better idea of what preprocessing we’ll have to do on the audio, if any, as well as what our biggest disadvantage is. As of now, our contingency plan is to preprocess our audio and in the worst case if latency does not hold, move to another pitch detection software, possibly implemented in C that Anita has researched.
Anna has been working on the web application and hopes to figure out our realistic latency by integrating the basic pitch detection algorithm to her application and see just how long it takes to display feedback as well as get the pitch from the backend. Risks here are being mitigated by integrating the backend one step at a time. Anna has been very diligent about breaking this arduous task up into first playing audio, then have a dummy python backend that can communicate, then record the users voice and send it to the backend to see how long it takes to come back with information. An obvious fallback here is to handle the pitch detection asynchronously instead of real-time as we’re hoping.
Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?
Currently, we added some use case requirements. As Anita stated in her report, the idea of relative vs. absolute pitch was brought up and we would like to accommodate relative pitch in our design. This comes at the cost of a potentially higher latency, but it also improves the accessibility of our product. Our product is marketed for beginner singers and thus every beginner should be able to use it, so we were quickly able to justify this change.
These costs will be mitigated moving forward by picking a good pitch detection algorithm that will limit the amount of preprocessing we’ll need to do on our recorded input.
This is also the place to put some photos of your progress or to brag about a component you got working.
This week, Anna has some preliminary graphing structure added to the webpage!
Additionally, Kelly was able to put together several mp3 files (as seen in her 2/25 report).
Enumerate how you have adjusted your team work assignments to fill in gaps related to either new design challenges or team shortfalls (e.g. a team member missing a deadline)
So far, our team has been working very hard to stay on schedule, but we did have to pivot our team’s structure quite a lot in light of no longer using a home grown pitch detection algorithm. Due this change, Anita no longer had a big task to work on and so we split up the pitch detection from the scoring and feedback process. This way, Kelly is focusing on pitch detection and accuracy, Anita is focused on feedback and scoring, and Anna is still focused on web application development. These tasks are all parallelizable, which we were very set on if we were going to get this project done on time. Additionally, the pitch detection algorithm testing managed to be a bigger task than we had originally planned for as Kelly needs to manually modify and in some cases create an audio file to compare the recorded vocals to. Thus, taking the work of the feedback and scoring off of Kelly was the right move for our team.
What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours). This week, I looked into midi files …