Melina’s Status Report for 04/26/2025
Schedule
Official Project Management Schedule Updates
- IN PROGRESS Backend Testing
- DOING Pitch analysis testing
- DOING CQ change analysis testing
- ‘testing’ in this context doesn’t really make sense, but I will be visually verifying that a trained opera singer has a CQ closer to the idealized CQ range than an amateur singer
- IN PROGRESS Inform about Change in CQ
- IN PROGRESS Final Demo Video
Pitch Analysis
- DOING: test all ‘Happy Birthday’ repertoires’ accuracy
- DONE: add function to output pitch based on the mouse’s position
- If I am understanding correctly, the frontend can pass the mouse cursor’s “x_pos”, which would represent a frequency value, to the backend. The function I wrote, get_pitch_label(), takes this value and converts it to a pitch note for the frontend to display as a small label next to the mouse cursor
- Improved pitch detection accuracy to 99.49% for all scales
CQ Analysis
- DONE: add function to output values of ‘ideal’ female and male ranges
- Instead of a function, I added constants (one for female, one for male) to proc_data.py
- This data was “recreated” based on the research papers by Howard. The papers summarize this data, but don’t explicitly give values for the summarized data. I had to create this summary with exact values and with slight modifications to fit our applications needs. For female data, I calculated two means (one based on all trained female singers, and a second based on only trained female singers with 10+ years of training) to compute CQ ranges. For male data, I made an executive decision to use the last session of the M1 (Male 1) test subject. The last sessions represents the recording session in which M1 had the most exposure to vocal training.
- While this tasks is computationally done, I will be meeting again with Susanna to ensure that the way I formatted the data is appropriate for the frontend to graph. I’m only concerned that because our research is limited to CQ measurements of certain pitches, that we will have to consider this when making the graph area look understandable to the reader. We can either fill in the “gaps” with a linear model, or represent only the concrete data in a “candlestick chart” style.
- TODO: Confirm ideal CQ data format with Susanna for frontend
Final Demo Video
- I recorded the footage I need for the final video
- TODO: Record video script narration
- TODO: Edit and publish video
Melina’s Status Report for 04/19/2025
Schedule
Official Project Management Schedule Updates
- IN PROGRESS Backend Testing
- DOING Pitch analysis testing
- DOING CQ change analysis testing
- ‘testing’ in this context doesn’t really make sense, but I will be visually verifying that a trained opera singer has a CQ closer to the idealized CQ range than an amateur singer
- IN PROGRESS Inform about Change in CQ
Pitch Analysis
- Adjusted intervals mapping frequency to pitch
- Added backend functions for extracting frequency given a pitch (to help with creating graph ticks for the frontend
- Tested pitch detection algorithm for all recordings of scales
- Results were 97.9% accurate
- DOING: test all ‘Happy Birthday’ repertoires’ accuracy
- DOING: add function to output pitch based on the mouse’s position over the audio player view of the pitch vs time container
CQ Analysis
- Implemented integration of CQ analysis into the backend
- I get the weighted average of every pitch-based time interval and output it into a list
- For the audio player, this list is given to the frontend along with corresponding time stamps and pitches for display
- For the graph of change in CQ over time, a function was added to process multiple data sets (representing distinct recordings) which further processes the data by mapping each recording’s distinct pitches to their averaged corresponding CQ.
Melina’s Status Report for 04/12/2025
Schedule
Official Project Management Schedule Updates
- IN PROGRESS Backend Testing
- DOING Pitch analysis testing
- NOT STARTED CQ change analysis testing
- IN PROGRESS Inform about Change in CQ
Pitch Analysis
There have been delays in receiving most of the pitch data from this week for testing. One important recording, C Major Scale by one of our opera singers, was obtained and testing yielded ~97.83% accuracy. This task is a little behind, so I will be following up with the team to get access to more crucial tests, such as Happy Birthday.
CQ Analysis
Currently the program now extracts CQ values at timestamps that correspond to the time a given pitch is from. The next step will be to implement a way to choose a set of CQ data based on an identifier (such as the name of a repertoire or the key of a scale for “name-less” warm up recordings). The idea is have a system that allows the front end to pass in these identifiers and receive the clean sets of data to pass on to DPG for graphing.
I conducted interviews with the vocalists this week to gain a better understanding of what changes in CQ they would find helpful to know about. The main takeaway was that they would like help in identifying “flutters” and “wobbles”. I am currently working on understanding how a CQ measurement might indicate one of these problems. There do not appear to be research papers on this specific topic, but if we take CQ measurement recordings of these problematic samples and compare them with ideal samples, there may be a pattern we can discern.
Regarding the issue with collecting audio and EGG signals at the same time, I have proposed transferring ownership of the Voce Vista access key to my computer which has a USB-B port that may solve the issue. Unfortunately, this might mean limiting Tyler’s ability to collect data. For the time being, we have decided to focus on implementing the first upcoming task mentioned in this section so that the front-end has something tangible to demo for the vocalists.
Melina’s Status Report for 03/29/2025
Schedule
Official Project Management Schedule Updates
- IN PROGRESS Backend Testing
- DOING Pitch analysis testing
- NOT STARTED CQ change analysis testing
- CANCELLED Streamline Voce Vista
- IN PROGRESS Inform about Change in CQ
Testing Pitch Analysis
We recently got a new interface for the Shure microphone which now allows us to record clear audio. Due to illness on the team, uploading the data and recording from Voce Vista was delayed, but became available today, so this upcoming week will focus on that.
Streamlining Voce Vista
We had previously thought that a command line was available to start the Voce Vista app, as it was mentioned in the documentation, but tech support confirmed that this functionality was deprecated. As a result this task has been cancelled. This means the general user flow would be to follow instruction on the recording page to open and utilize Voce Vista instead of pressing a record button on our app directly. There would also be instructions to export data. This process could be facilitated by instructing the user to use 1-2 keyboard shortcuts.
Inform about changes to CQ
Tyler and I are brainstorming what it functionally means to inform about changes in CQ. At the very minimum, our app graphs CQ average per pitch over time, which is being handled by Susanna in the frontend. What Tyler and I are thinking about, is what additional information about CQ changes over time could be provided. We want to provide useful information that is beyond what a user could simply infer from just looking at the original graph. There are two areas I could see additional analysis come in:
1. More analysis (specifics currently undetermined) is added to CQ changes over time for the warm-up.
2. A CQ analysis is added to the repertoire recording. Currently we plan to simply list the measured CQ on top of the audio signal something like this:
Perhaps by making use of DPG and our pitch analysis algorithm, we can graph the average CQ for each pitch detected in the repertoire. Maybe we could also show them the difference between a CQ average for a pitch in the warm-up versus the repertoire. The main question here is how this would be useful for the user. Would they be informed enough to make a reasonable decision on vocal technique based on this additional analysis? Would the resulting decision have negative consequences?
Melina’s Status Report for 3/22/2025
Schedule
Official Project Management Schedule Updates
- COMPLETED Pitch Analysis
- IN PROGRESS Streamline Voce Vista
Personal Task Updates
- DONE Propose modification to Pitch Analysis Testing
- DONE Add normalization support to pitch analysis of vibrato
- DOING Draft code for streamlining Voce Vista
- DOING Test pitch analysis accuracy
Pitch analysis has been implemented, however modifications are expected to follow for the sake of accuracy. This will be determined once we formally test the algorithm. Currently, informal tests have been completed in which we recorded the vocalists singing a C Major scale and “Happy Birthday”. I call this an informal test, because for these recording sessions, we were not able to utilize the XLR microphone. When connected to the EGG, our laptop could not properly detect and record from the microphone. As a result, we decided to record the vocalists from our laptop’s built-in microphone this week. While this allowed me to roughly test the pitch analysis program, I do not expect this to be a fully accurate representation of the algorithm’s accuracy. For a C Major scale, for example, the resulting analysis was as shown below:
We are generally on schedule, but will need CQ data to be extracted from Voce Vista this upcoming week to stay on track.
Team Status Report for 3/15/2025
Due to illness going around the team, some frontend tasks and data exportation have been delayed, but important strides were made in pitch analysis. Our first major data collection event also took place on Wednesday when we took audio recordings of the vocalists for the purpose of testing pitch analysis. Recordings included a variety of styles inlcuding vibrato, staccato, with piano, and without piano. An important observation was made that our opera vocalists naturally lean toward singing vibrato unless specifically instructed to sing staccato. This motivated the extension of the current pitch analysis to support normalization of data in order to extract the fundamental frequency of a vibrato note.
The main challenges we face this upcoming week including extracting and exporting CQ data long-term, adding pitch analysis support for vibrato, streamlining Voce Vista, and adding frontend views for playback and CQ data. Note that streamlining Voce Vista is a new task added to the official schedule and assigned to Melina.
Melina’s Status Report for 3/15/2025
Schedule
Official Project Management Schedule Updates
- IN PROGRESS BLOCKED Pitch Analysis
- COMPLETED Match Microphone Data to EGG
- IN PROGRESS Streamline Voce Vista
Personal Task Updates
- DONE Draft code for pitch analysis
- DOING Propose modification to Pitch Analysis Testing
- DOING Add normalization support to pitch analysis of vibrato
- DOING Draft code for streamlining Voce Vista
Pitch analysis is slightly behind, but very close to being done. Other that that, tasks are on-time. It is important to note that the task “Match Microphone Data to EGG” was marked complete ahead of schedule because Voce Vista already outputs the data we need for matching up time and pitch long-term, but only outputs CQ data in the short-term.
Pitch Analysis
Tyler and I collected some recordings of one of the vocalists singing scales and “Happy Birthday” in different styles (vibrato, staccato, with piano, without piano). Shortly after this, almost the whole team was feeling under the weather, so there has been some delays in getting this data uploaded to the drive. As soon as I have access to the new data, I will be able to implement the last part of pitch analysis, support for vibrato. Prof. Sullivan and I believe that if I normalize the data appropriately, I will be able to extract the fundamental frequency around which the pitch varies when a vocalist is singing vibrato. Supporting vibrato would be great for making our product useful for opera singers, who we observed to sing vibrato naturally. Even when asked to sing staccato, occasional slips into vibrato occurred, so it would be in the interest of minimizing over-self-consciousness that we provide support for vibrato in our pitch detection. My goal is to have this support complete by the next status report. A modification to pitch analysis testing was also proposed to involve the vocalists and vocal coach as described in the previous status report, “our testing would likely be modified to include having a vocalist sing a series of notes, such as “Happy Birthday” and have the vocal coach verify that that the pitch detected agrees with their perception of the pitch for 90% of the notes sung.” This proposal is awaiting feedback.
Match Microphone Data to EGG
As stated above, this task is being marked as complete and the team is instead focusing on actually extracting and exporting CQ data long-term in the same format as time and pitches. Tyler and I are working together on this, but he is taking the lead on communicating with Voce Vista support for learning how to export this data as part of his task “Integrate EGG Data”. If there is further support needed to match CQ to time stamps, this tasks would reopen.
Streamlining Voce Vista
Adding any support to streamline the use of Voce Vista would be in the interest of our product to make the analysis process easy for our users. Voce Vista does not appear to support the command line, but it has an extensive list of configurable keyboard shortcuts. These shortcuts could be conditionally triggered from a python script with the keyboard library. This week, I will add code with functions to trigger Voce Vista functionalities including opening the app and immediately recording upon startup, stopping a recording, and exporting time and pitch data to a specific folder in the repository.
Melina’s Status Report for 3/1/2025
Schedule
Official Project Management Schedule Updates
- COMPLETE Design Report
- IN PROGRESS Pitch Analysis
- NOT STARTED Match Microphone Data to EGG
Personal Task Updates
- DONE Draft Design Review Report with the team
- DONE Ensure the team has a repository set up along with agreement on code style/organizational expectations
- DOING Draft code for pitch analysis
- TODO Propose modification to Pitch Analysis Testing
My tasks are on time, and given that nothing new was currently scheduled, I have Week 8 to complete pitch analysis.
Pitch Analysis
An initial attempt has been made at utilizing Librosa for pitch analysis. I learned how to load an audio file and extract basic information such as tempo, frequencies and their magnitudes. The current issue is that the pitch detection algorithm outputs a lot of frequencies, with some that appear to be noise. Overall extracting frequencies at the correct times is difficult, however, after reading Tyler’s updates, I saw that Voce Vista may already output the frequencies in a cleaner manner. This would help the algorithm more accurately map those frequencies to a note based on a standardized chart. The algorithm will have to take into account that there is some natural variance in pitch for the human voice compared to instruments. For example, the standardized chart marks G4 at 392 Hz and G4# at 415.3. An appropriate range has to be considered to distinguish adjacent notes. I’m currently thinking of approaching this by providing some slack in the range of frequencies that map to a note. This means our testing would likely be modified to include having a vocalist sing a series of notes, such as “Happy Birthday” and have the vocal coach verify that that the pitch detected agrees with their perception of the pitch for 90% of the notes sung. Proposing this modification for testing to the team has been noted as a new personal task.
Team Status Report for 2/22/2025
VoceVista
This week, we got our request approved for purchasing VoceVista, and began to learn how to use it. The one lingering concern is that we only have one copy of VoceVista at present, in use on Tyler’s laptop. At the moment, we think that this will be sufficient for our purposes: Tyler can use his copy of VoceVista to generate output files from tests with the EGG, then send these files to Melina and Susanna to be used as test inputs as they develop code. Additionally, there’s a 1-month trial version that we can use at key times if necessary.
However, we’re planning to reassess as the project develops, as it may become necessary for Melina, in particular, to have more direct access to VoceVista while she works on the backend. Additionally, we don’t currently have any major purchases in mind that will be necessary for the remainder of the project, so a second VoceVista access code might be a good investment if no more pressing costs come up. In the coming weeks, we’ll be paying attention to any potential for further costs, as well as the extent that a single access to VoceVista is a burden.
DearPyGUI
While TKinter is a widely-used and versatile framework for Python app-building, we’ve been discussing other options that are more versatile for the plotting functionality that our app will require. Specifically, DearPyGUI has both a powerful graphic display and strong backend functionalities that tie in closely with the frontend graphics. Though Susanna has made a start on working on the TKinter app, it’s not too late to change the framework, but if we’re going to do so we ought to do it as soon as possible. We will be testing DearPyGUI and making an assessment by the end of the week.
Pitch Analysis
We realized that the tasks in our schedule were a bit out of order– in particular, the pitch analysis task occurred before we received our microphone to record audio. As we have now acquired our microphone and cable, we bumped our pitch analysis task to next week.