Posted on

Team Status Report for 3/8/25

This past week, we mostly focused on finishing up our design review by ensuring that the language was specific enough and concise. Additionally, we focused on adding in clear graphics, like pseudo graphics and diagrams to help convey the information. In addition to this, we have also met with Professor Almaraza and confirmed the use case requirements and gained their opinion on the current workflow. From this, we also got three flutists to sign up for testing for our project and will now be working on getting them additionally sign up for the conjoined mini course. In terms of the project, we have a more clear understanding of how to implement audio segmentation after some research and discussing the concept with professor Sullivan and look towards really finishing this portion up this next week. 

Overall, we are currently on track, though may run into the some issues with the audio segmentation as this will be the most difficult aspect of our project.

Part A (Grace): Our product solution will meet the global factors for those who are not as technologically savvy by making the project as easy to understand as possible. Already this project is significantly decreasing the strain of composing your own music by eliminating the need for individuals to know exact lengths of notes, pitches, etc of when they are composing music and simplifying the process by decreasing the amount of time it takes to transcribe. In addition to this, the individual will only have to upload an mp4 recording of them playing before getting transcribed music, as we will be handling all the backend aspects of this. As such, even the technologically unsavvy should be able to use this application. Furthermore, we aim to make the UI user friendly and easy to read. 

In addition, we aim to make this usable in other environments, not just an academic one, by filtering out the outside noise to allow users to be able to use the application in even noisy settings. As mentioned in our design reviews, we will be testing this application in multiple different settings to hopefully encompass the different environments this website would be used globally. 

Part B (Shivi): Our project can make a cultural impact by allowing people to pass down music that lacks standardized notation. For instance, the traditional/folk tunes (such as the Indian bansuri or Native American flute) are often played by ear and likely to be lost over time, but our project can help transcribe such performances, allowing them to be preserved over multiple generations. This would also help to increase access to music for people from different cultures, promoting cross-cultural collaboration. 

Part C (Deeya): Write on Cue is addressing environmental factors by encouraging a more sustainable system of digital transcription over printed notation – reducing paper usage. Also digital transcription allows musicians to learn, compose, and practice remotely, reducing the need for physical travel to lessons, rehearsals, or recording sessions. By reducing transportation energy and paper consumption, it helps make our product more environmentally-friendly. Also, instead of relying on large, energy-intensive AI models, we are going to use smaller, more efficient models trained specifically for flute music, which will help reduce computation time and power consumption. We will look into techniques like quantization to help speed up inference.

Posted on

Grace’s Status Report for 3/8/25

Last week, I primarily worked on the design review document, refining the finer details. Additionally, the conversation we had during the design presentation was useful as we decided that the noise filtering features might be excessive, especially with the microphone being so close to where the signal will be coming from. As our microphone has just recently come in, we are excited to experiment with this process and test these in environments that flutists commonly compose in, like their personal rooms (where there might be slight background noise) and studios (virtually no noise). Hopefully with the calibration step, we can eliminate excessive noise filtering and decrease the amount of time it takes to get the sheet music to the user. Furthermore, after meeting with Professor Sullivan last week, we have a better idea of implementing audio segmentation, deciding to focus on the RMS rather than just the peaks in amplitude for note beginnings, so we plan on implementing a sliding RMS window of around 10 ms and looking for peaks there. After creating these segmentations, I plan to implement my module of rhythm detection here since we cannot just use the length of the segmentation as there might be a rest in the segmentation. Overall we are currently on track, but this week we expect to run into more issues as audio segmentation will most likely be the hardest aspect of our project.

Finally, we are excited for how our collaboration with the music department will look as many seem interested. We will be reaching out to the flutists this week to get them registered for the mini as well.  

Posted on

Grace’s Status Report For 2/22/25

This week, I mostly focused on working on our design review presentation and our design proposal. Since this week I was the one presenting, I focused on making sure our slides were ready and rehearsed, and ensured that I could present all the information from my team mate’s slides as well as limiting words to make our presentation less word heavy. In class, we listened to the different presentations and gave feedback. This was helpful since professor Sullivan mentioned that some of our filtering for noise suppression might be excessive, which would slow down our processing time and lengthen our latency, so we will continue with experimenting using just the bandpass and see if the other filtering methods are necessary. I further experimented with using different thresholds and filters to better isolate the rhythm of single notes. 

For this week, I will be working on my sections of the design review paper as well as doing further research on audio segmentation because depending on how this is working will determine how I will implement rhythm detection. I will be meeting with Shivi to work on this aspect. Our project is currently on schedule but other tasks might have to be pushed back with the introduction of audio segmentation. I hope to get this (audio segmentation + rhythm detection) working by the week after spring break at the latest.

Posted on

Grace’s Status Report for 2/15/25

This week, I worked on creating the rhythm detection algorithm. We first practiced by simply writing into a midi file, using the mido library in python, and then uploading the output into musescore so we could see what the sheet music generation looked like. 

We are trying to get the bare bones aspect of the project working, so we did a few different recordings, including the metronome alone, someone playing a D on the flute with the metronome in the background and no other sound, and then someone playing a D on the flute with some background noise (people talking). This helps us test with just detecting a note with the clear recording, but also experiment with noise suppression, which Shivi is working on. 

(what the isolated signals for the metronome and the flute note look like) 

After analyzing what these frequencies look like after doing a fourier transform on them, I isolated the signal by using a filter to filter out all other frequencies than the pitch of the note played and calculated the duration of the notes, using the inputted bpm. However, with audio recordings, there tends to be a lot of variation in the sound quality, creating a lot of peaks within the wave. This originally made my code think there were multiple different notes being played since I was trying to calculate it by the peaks. After analyzing the signal further, I migrated to using 20% of the max amplitude to use as a threshold to calculate the duration of a note. I then transcribed this into a midi file and uploaded it to musescore to look at the results. Though it is still not accurate for the rhythm, I am hopeful that this will be working soon and plan on using a sliding window filter in future testing to reduce the number of peaks and noise. 

(what is currently being transcribed to musescore, should just be a signal note so will need to reassess my threshold value) 

My current progress is on schedule. This next week I hope to get the rhythm detection accurately working for a whole note, quarter note, and a half note for the same note at the very least. Hopefully, I will be able to detect switches in notes soon as well.

Posted on

Team Status Report for 02/08/25

As we had proposal presentations this week, we worked hard on finishing up our slides, ensuring that they were done far enough in advance that Ankit, our TA, would be able to give us feedback on our schedule. Here, Ankit had mentioned the possibility of converting our hardware systems (like the microcontroller of an arduino) be done solely in software instead, as it would function a lot faster. We are currently considering this option: Since we would ideally like to convert this system into real time, it would be best for faster processing. However, this could result in changes on how we approach tasks, like rhythm detection. We are planning on reaching out to Ankit again to talk this over further. 

Last week, we also meet with Professor Dueck and other musicians to discuss what our project looks like and how the music department could contribute to our project, such as allowing us to work in her studio to test the flutes in a relatively noiseless environment, which would be best for a bare bones working project. Additionally, she connected us with Professor Almarza, who will be helping us find some flutists to help test our project.

After this, we experimented with looking at some straight tone flute signals and seeing how this pitch would appear in Matlab. This is to get more insight in getting a bare bones project up and working.

Currently, our most significant risk would be switching the project and having unforeseen consequences and then having to backtrack to the hardware idea, which is a little more fleshed out due to past project references. These risks could be managed by discussing this further with our TA and staff, like Professor Sullivan. As such, this might pose a possible change to the existing design, specifically the system spec, to help with the speed. Overall, we feel that we are on track and excited to see where our project tasks us as well as work collaboratively with CMU musicians to get their feedback throughout the process.

Posted on

Grace Status Report for 02/08/25

This week, we primarily focused on finishing our proposal slides. We made sure to get them done in advance so our TA, Ankit, would be able to give us feedback on our schedule and our parts. In class, we listened to the different presentations and gave feedback. It was helpful to see all the presentations in our section, since most of them also related to signal processing, and it gave us inspiration on how we might approach handling our signals from the flute. 

During the presentation, Ankit gave us some insightful feedback on whether we should continue using hardware for the signal processing, which would entail using a microcontroller and possibly an op amp, or move to entirely software. It is true that most likely using solely software would help with the real time aspect of the project, but I’m not entirely sure how this would end up translating for the rhythm detection, which I am tasked with. Originally, our plan was to include a blinking light on a physical device/hardware and that would allow people to maintain a steady bpm (essentially providing a metronome). With moving to software, we could implement a similar feature on the web application but I’m not sure how accurately we would be able to detect rhythm since our original plan was to time it with how long it would take to send to the web app via serial. I would definitely need to do some more research on how to record and detect rhythm. 

Our plan from here on out is to divide into our scheduled task as we are currently on schedule, with Shivi and I primarily working on the signal processing side of the project and Deeya doing research and beginning to work on the web app. Our first next step would be reaching out to Ankit, or any other specialist TA in signal processing, to ask which would work best for us in terms of hardware vs software.

Looking forward, I hope to have a solid game plan and have implemented the bare bones of the rhythm detection for the signal processing and work on integrating this information with Shivi to put into our midi file within the next two weeks. Furthermore, we hope to meet with some flutists from CFA and get their feedback on how they would like the website to look and how they would like the machine to function.