Schedule

Official Project Management Schedule Updates

• IN PROGRESS Backend Testing
  • DOING Pitch analysis testing
  • NOT STARTED CQ change analysis testing
• IN PROGRESS Inform about Change in CQ

Pitch Analysis

There have been delays in receiving most of the pitch data from this week for testing. One important recording, C Major Scale by one of our opera singers, was obtained and testing yielded ~97.83% accuracy. This task is a little behind, so I will be following up with the team to get access to more crucial tests, such as Happy Birthday.

CQ Analysis

Currently the program now extracts CQ values at timestamps that correspond to the time a given pitch is from. The next step will be to implement a way to choose a set of CQ data based on an identifier (such as the name of a repertoire or the key of a scale for “name-less” warm up recordings). The idea is have a system that allows the front end to pass in these identifiers and receive the clean sets of data to pass on to DPG for graphing.

I conducted interviews with the vocalists this week to gain a better understanding of what changes in CQ they would find helpful to know about. The main takeaway was that they would like help in identifying “flutters” and “wobbles”. I am currently working on understanding how a CQ measurement might indicate one of these problems. There do not appear to be research papers on this specific topic, but if we take CQ measurement recordings of these problematic samples and compare them with ideal samples, there may be a pattern we can discern.

Regarding the issue with collecting audio and EGG signals at the same time, I have proposed transferring ownership of the Voce Vista access key to my computer which has a USB-B port that may solve the issue. Unfortunately, this might mean limiting Tyler’s ability to collect data. For the time being, we have decided to focus on implementing the first upcoming task mentioned in this section so that the front-end has something tangible to demo for the vocalists.

Schedule

Official Project Management Schedule Updates

• IN PROGRESS Backend Testing
  • DOING Pitch analysis testing
  • NOT STARTED CQ change analysis testing
• CANCELLED Streamline Voce Vista
• IN PROGRESS Inform about Change in CQ

Testing Pitch Analysis

We recently got a new interface for the Shure microphone which now allows us to record clear audio. Due to illness on the team, uploading the data and recording from Voce Vista was delayed, but became available today, so this upcoming week will focus on that.

Streamlining Voce Vista

We had previously thought that a command line was available to start the Voce Vista app, as it was mentioned in the documentation, but tech support confirmed that this functionality was deprecated. As a result this task has been cancelled. This means the general user flow would be to follow instruction on the recording page to open and utilize Voce Vista instead of pressing a record button on our app directly. There would also be instructions to export data. This process could be facilitated by instructing the user to use 1-2 keyboard shortcuts.

Inform about changes to CQ

Tyler and I are brainstorming what it functionally means to inform about changes in CQ. At the very minimum, our app graphs CQ average per pitch over time, which is being handled by Susanna in the frontend. What Tyler and I are thinking about, is what additional information about CQ changes over time could be provided. We want to provide useful information that is beyond what a user could simply infer from just looking at the original graph. There are two areas I could see additional analysis come in:

1. More analysis (specifics currently undetermined) is added to CQ changes over time for the warm-up.

2. A CQ analysis is added to the repertoire recording. Currently we plan to simply list the measured CQ on top of the audio signal something like this:

Perhaps by making use of DPG and our pitch analysis algorithm, we can graph the average CQ for each pitch detected in the repertoire. Maybe we could also show them the difference between a CQ average for a pitch in the warm-up versus the repertoire. The main question here is how this would be useful for the user. Would they be informed enough to make a reasonable decision on vocal technique based on this additional analysis? Would the resulting decision have negative consequences?

Schedule

Official Project Management Schedule Updates

• COMPLETED Pitch Analysis
• IN PROGRESS Streamline Voce Vista

Personal Task Updates

• DONE Propose modification to Pitch Analysis Testing
• DONE Add normalization support to pitch analysis of vibrato
• DOING Draft code for streamlining Voce Vista
• DOING Test pitch analysis accuracy

Pitch analysis has been implemented, however modifications are expected to follow for the sake of accuracy. This will be determined once we formally test the algorithm. Currently, informal tests have been completed in which we recorded the vocalists singing a C Major scale and “Happy Birthday”. I call this an informal test, because for these recording sessions, we were not able to utilize the XLR microphone. When connected to the EGG, our laptop could not properly detect and record from the microphone. As a result, we decided to record the vocalists from our laptop’s built-in microphone this week. While this allowed me to roughly test the pitch analysis program, I do not expect this to be a fully accurate representation of the algorithm’s accuracy. For a C Major scale, for example, the resulting analysis was as shown below:

We are generally on schedule, but will need CQ data to be extracted from Voce Vista this upcoming week to stay on track.

Schedule

Official Project Management Schedule Updates

• IN PROGRESS BLOCKED Pitch Analysis
• COMPLETED Match Microphone Data to EGG
• IN PROGRESS Streamline Voce Vista

Personal Task Updates

• DONE Draft code for pitch analysis
• DOING Propose modification to Pitch Analysis Testing
• DOING Add normalization support to pitch analysis of vibrato
• DOING Draft code for streamlining Voce Vista

Pitch analysis is slightly behind, but very close to being done. Other that that, tasks are on-time. It is important to note that the task “Match Microphone Data to EGG” was marked complete ahead of schedule because Voce Vista already outputs the data we need for matching up time and pitch long-term, but only outputs CQ data in the short-term.

Pitch Analysis

Tyler and I collected some recordings of one of the vocalists singing scales and “Happy Birthday” in different styles (vibrato, staccato, with piano, without piano). Shortly after this, almost the whole team was feeling under the weather, so there has been some delays in getting this data uploaded to the drive. As soon as I have access to the new data, I will be able to implement the last part of pitch analysis, support for vibrato.  Prof. Sullivan and I believe that if I normalize the data appropriately, I will be able to extract the fundamental frequency around which the pitch varies when a vocalist is singing vibrato. Supporting vibrato would be great for making our product useful for opera singers, who we observed to sing vibrato naturally. Even when asked to sing staccato, occasional slips into vibrato occurred, so it would be in the interest of minimizing over-self-consciousness that we provide support for vibrato in our pitch detection. My goal is to have this support complete by the next status report. A modification to pitch analysis testing was also proposed to involve the vocalists and vocal coach as described in the previous status report, “our testing would likely be modified to include having a vocalist sing a series of notes, such as “Happy Birthday” and have the vocal coach verify that that the pitch detected agrees with their perception of the pitch for 90% of the notes sung.” This proposal is awaiting feedback.

Match Microphone Data to EGG

As stated above, this task is being marked as complete and the team is instead focusing on actually extracting and exporting CQ data long-term in the same format as time and pitches. Tyler and I are working together on this, but he is taking the lead on communicating with Voce Vista support for learning how to export this data as part of his task “Integrate EGG Data”. If there is further support needed to match CQ to time stamps, this tasks would reopen.

Streamlining Voce Vista

Adding any support to streamline the use of Voce Vista would be in the interest of our product to make the analysis process easy for our users. Voce Vista does not appear to support the command line, but it has an extensive list of configurable keyboard shortcuts. These shortcuts could be conditionally triggered from a python script with the keyboard library. This week, I will add code with functions to trigger Voce Vista functionalities including opening the app and immediately recording upon startup, stopping a recording, and exporting time and pitch data to a specific folder in the repository.

Schedule

Official Project Management Schedule Updates

• COMPLETE Design Report
• IN PROGRESS Pitch Analysis
• NOT STARTED Match Microphone Data to EGG

Personal Task Updates

• DONE Draft Design Review Report with the team
• DONE Ensure the team has a repository set up along with agreement on code style/organizational expectations
• DOING Draft code for pitch analysis
• TODO Propose modification to Pitch Analysis Testing

My tasks are on time, and given that nothing new was currently scheduled, I have Week 8 to complete pitch analysis.

Pitch Analysis

An initial attempt has been made at utilizing Librosa for pitch analysis. I learned how to load an audio file and extract basic information such as tempo, frequencies and their magnitudes. The current issue is that the pitch detection algorithm outputs a lot of frequencies, with some that appear to be noise. Overall extracting frequencies at the correct times is difficult, however, after reading Tyler’s updates, I saw that Voce Vista may already output the frequencies in a cleaner manner. This would help the algorithm more accurately map those frequencies to a note based on a standardized chart. The algorithm will have to take into account that there is some natural variance in pitch for the human voice compared to instruments. For example, the standardized chart marks G4 at 392 Hz and G4# at 415.3. An appropriate range has to be considered to distinguish adjacent notes. I’m currently thinking of approaching this by providing some slack in the range of frequencies that map to a note. This means our testing would likely be modified to include having a vocalist sing a series of notes, such as “Happy Birthday” and have the vocal coach verify that that the pitch detected agrees with their perception of the pitch for 90% of the notes sung. Proposing this modification for testing to the team has been noted as a new personal task.

Schedule

Official Project Management Schedule Updates

• COMPLETED Design Presentation
• IN PROGRESS Design Report
• IN PROGRESS Pitch Analysis

My progress from goals from last week. Any DOING tasks will be rolled over as goals for next week.

• DONE Present Design Review Slides
• DONE Drafting wireframes for the frontend with feedback from Susanna
• DOING BLOCKED Draft code for pitch analysis
• DOING Draft Design Review Report with the team
• DOING Ensure the team has a repository set up along with agreement on code style/organizational expectations

Most of these tasks were ahead of schedule, so we’re not worried about them being in progress. The official schedule would suggest Pitch Analysis should have been done this week, but this was misleading; I will elaborate below in Pitch Analysis.

Design Report

We are in the process of drafting the Design Report.

I have shared a Google Doc with sections for us to draft content in, as well as setting up the latex file we ultimately use to format our final design report.

I have drafted a revised block diagram that integrates our hardware and software systems. This reflects a proposal I made to the team that changes our software design; I propose we use Dear PyGUI instead of TKinter. This tool is well documented and provides more

This diagram reflects a proposal I made to the team that modifies our software design; I propose we use Dear PyGUI instead of TKinter. This toolkit is well documented and provides more support for app development and data plotting which is an important aspect of our project for visualizing users’ CQ time series analysis.

I am also currently drafting the following sections

• Design Requirements
• Software Design Specification or Subsystem
• Tests for Use-Case Specification Software
• Related Work

Pitch Analysis

I realized our scheduling of “Pitch Analysis” was slightly out of order since it should be after we have a processed microphone signal to work with. Tyler very recently acquired the full-version authentication code for VoceVista, so as soon as  Tyler sends me at least 1 digital audio signal recording, this task will be considered unblocked and I can continue working on it this upcoming week.

Git Repository

I confirmed with Fiona that we can set up our own repository, and prepared a discussion with the team about repository management expectations and code style. This discussion will be brought up on Wednesday since we are currently focusing on drafting the Design Report. By Wednesday I will at least have the repository created so it’s ready to clone.

EGG Hunt

• Picked up all hardware components associated with the EGG EG2-PCX from Prof. Helou, along with the corresponding digital user manual
• This includes a simple microphone which we plan to replace with a more sophisticated microphone

Ground-Truth Research

• Annotated reading “Results from a pilot longitudinal study of electrolaryngographically derived closed quotient for adult male singers in training” (David Howard)
• Annotated reading “The Physiology of Vocal Damping: Historical Overview and Descriptive Insights in Professional Singers” (Marco Fantini)
• Annotated reading “Musical Theater and Opera Singing—Why So
  Different? A Study of Subglottal Pressure, Voice Source, and Formant Frequency Characteristics” (Eva Björkner)
• Concluded that identifying a “proper” CQ would not be the most appropriate solution approach since the number of years in singing experience and genre would be required for such a metric. This data is limited at this time and would not be reliable for the purposes of our project
  • Because of this we have adjusted our scheduled tasks to reflect this change
    • Understand/identify improper CQ CANCELLED – we want to stray away from defining a universal truth for what a proper CQ is, instead focusing on helping vocalists track and understand their formant tuning
    • Inform about change in CQ ← changed from detect/warn about improper CQ
• Proposed our project solution approach should shift from warning of “improper” CQ to providing more flexible analysis tools (2) for tracking CQ over time
  • Analysis tool 1: Allowing user to view their CQ at a given moment for a specific recording playback
  • Analysis tool 2: Providing user with a visual representation of an evaluation of their CQ range over time
    • This would be approached by asking the user to record a controlled vocal exercise, such as an octave warmup that covers their tessitura (comfortable vocal range)
    • The CQ ranges for these controlled (constant) exercises can be summarized visually over time as suggested by David Howard’s graph of idealized changes in CQ
    • 
  • Proposed our use case and solution approach should shift to focus on advanced vocalists in one genre, opera singers
    • CQ can be significantly more difficult to measure for untrained singers, in fact, David Howard had to completely throw out some data samples from untrained singers due to unreliable CQ measurements
    • Unreliable CQ measurements are detrimental to our project, as an incorrect analysis could mislead a vocalist to make unhealthy decisions
    • CQ has also been found to vary significantly with genre, and as of now, we only have guaranteed access to opera singers
  • Created a ground-truth metric
    • EGG passes calibration test with the laryngeal simulator before and after usage
      • This is handy dandy calibration hardware component that came with the EGG itself thanks to Prof. Helou
    • Use built-in Larynx Tracking & Signal-Strength indicators
    • CQ measurement must be at least 20% to be considered detected

Schedule

• COMPLETED Acquired sensor
• COMPLETED Create a “Ground-Truth” metric

My scheduled tasks are on time, and have been slightly adjusted as described above (I will reiterate below)

• Understand/identify improper CQ CANCELLED – we want to stray away from defining a universal truth for what a proper CQ is, instead focusing on helping vocalists track and understand their formant tuning
•  Inform about change in CQ ← changed from detect/warn about improper CQ

Next Steps

My goals for this next week include the following deliverables

• Present Design Review Slides
• Draft Design Review Report with the team
• Ensure the team has a repository set up along with agreement on code style/organizational expectations
• Drafting wireframes for the frontend with feedback from Susanna
• Draft code for pitch analysis