Posted on

Shivi’s Status Report for 02/08/25

This week, I worked with Grace and Deeya to finish our proposal slides, where we included a very high-level workflow of our design. Completing the proposal slides gave us a better idea of the amount of work we need to do, and the three of us met up to generate some flute audio recordings. Since I am tasked with pitch detection, as an experiment, I wrote a basic Python script that performs a fast Fourier Transform on singular notes so that we could examine the frequencies associated with a few notes from the B flat major scale:

Here, we can see the fundamental frequencies/harmonics associated with each note, a property that we will leverage to determine which note is being played in the audio. After proposal presentations, we thought about some feedback from our TA (Ankit) and realized that we need to think more about software-hardware tradeoffs in our design. Initially, we were keen on having a hardware component in our project (having taken/taking 18341 and 18349 as well as seeing similar projects from the past doing this), but it seems that it may be cleaner/more efficient to simply perform certain tasks purely in software. For instance, our initial design included performing FFT using the microcontroller, but it will definitely be more efficient to perform it on a laptop CPU. These are some of my thoughts for a revised design (at least on the signal processing side) based on some independent research:

  • Signal Processing 
    • Use microphone to capture flute audio
      • Suggested mic: InvenSense ICS-43434, a MEMS microphone with digital output. Can be mounted close to the flute’s embouchure hole and does not require any sort of PCB soldering. We also have the option to 3D print a custom clip to attach it to the flute for optimal placement.
      • Send audio to microcontroller via I2S (Inter-IC sound interface)
      • Microcontroller converts PDM (Pulse Density Modulation) to PCM (Pulse Code Modulation). Some suggested microcontrollers with built-in PDM support: RPi RP2040, STM32 (more suited for high-end tasks and higher performance so might not be necessary)
    • In software, do pitch detection: 
      • Apply additional digital filtering to the PCM signal: noise suppression, bandpass filtering, adaptive filtering
      • Apply Fast Fourier Transform to detect flute frequencies, map frequencies to flute notes
      • Use moving average filter (ex: Kalman filter) to smooth out pitch detection
    • In software, do note length detection:
      • Use Peak Tracking in Frequency Domain (more computationally expensive than methods like time-domain envelope filtering and requires harmonic filtering to avoid detecting overtones, but less sensitive to volume variations and more accurate in noisy environments)
      • Detect note length: note is considered ongoing if the peak frequency remains stable. If the peak disappears or shifts significantly, the note has ended.
    • MIDI: store the note frequencies, durations in a MIDI format. Then, generate a MIDI Note On message when note starts (0x90 message), MIDI Note Off message when note ends (0x80). Use duration to check note type (eighth, quarter, half, whole note, etc)
    • Use MuseScore API to upload MIDI file and display sheet music on web app

For the coming week, we plan to flesh the design out more and work on our low-level design with other important details such as BPM detection, metronome, and integration with webapp. We also aim to make a list of any inventory/purchase items we will need.

Posted on

Team Status Report for 02/08/25

As we had proposal presentations this week, we worked hard on finishing up our slides, ensuring that they were done far enough in advance that Ankit, our TA, would be able to give us feedback on our schedule. Here, Ankit had mentioned the possibility of converting our hardware systems (like the microcontroller of an arduino) be done solely in software instead, as it would function a lot faster. We are currently considering this option: Since we would ideally like to convert this system into real time, it would be best for faster processing. However, this could result in changes on how we approach tasks, like rhythm detection. We are planning on reaching out to Ankit again to talk this over further. 

Last week, we also meet with Professor Dueck and other musicians to discuss what our project looks like and how the music department could contribute to our project, such as allowing us to work in her studio to test the flutes in a relatively noiseless environment, which would be best for a bare bones working project. Additionally, she connected us with Professor Almarza, who will be helping us find some flutists to help test our project.

After this, we experimented with looking at some straight tone flute signals and seeing how this pitch would appear in Matlab. This is to get more insight in getting a bare bones project up and working.

Currently, our most significant risk would be switching the project and having unforeseen consequences and then having to backtrack to the hardware idea, which is a little more fleshed out due to past project references. These risks could be managed by discussing this further with our TA and staff, like Professor Sullivan. As such, this might pose a possible change to the existing design, specifically the system spec, to help with the speed. Overall, we feel that we are on track and excited to see where our project tasks us as well as work collaboratively with CMU musicians to get their feedback throughout the process.

Posted on

Grace Status Report for 02/08/25

This week, we primarily focused on finishing our proposal slides. We made sure to get them done in advance so our TA, Ankit, would be able to give us feedback on our schedule and our parts. In class, we listened to the different presentations and gave feedback. It was helpful to see all the presentations in our section, since most of them also related to signal processing, and it gave us inspiration on how we might approach handling our signals from the flute. 

During the presentation, Ankit gave us some insightful feedback on whether we should continue using hardware for the signal processing, which would entail using a microcontroller and possibly an op amp, or move to entirely software. It is true that most likely using solely software would help with the real time aspect of the project, but I’m not entirely sure how this would end up translating for the rhythm detection, which I am tasked with. Originally, our plan was to include a blinking light on a physical device/hardware and that would allow people to maintain a steady bpm (essentially providing a metronome). With moving to software, we could implement a similar feature on the web application but I’m not sure how accurately we would be able to detect rhythm since our original plan was to time it with how long it would take to send to the web app via serial. I would definitely need to do some more research on how to record and detect rhythm. 

Our plan from here on out is to divide into our scheduled task as we are currently on schedule, with Shivi and I primarily working on the signal processing side of the project and Deeya doing research and beginning to work on the web app. Our first next step would be reaching out to Ankit, or any other specialist TA in signal processing, to ask which would work best for us in terms of hardware vs software.

Looking forward, I hope to have a solid game plan and have implemented the bare bones of the rhythm detection for the signal processing and work on integrating this information with Shivi to put into our midi file within the next two weeks. Furthermore, we hope to meet with some flutists from CFA and get their feedback on how they would like the website to look and how they would like the machine to function.