Author: aruiz2

This week I looked into what best parameters might be used for the rhythm processor to detect the onset peaks in an audio signal. The function find_peaks from SciPy can take in several parameters and we think that the best parameters to be utilized for our rhythm processor will be the peak distance parameter and the height parameter.

The peak distance parameter will be set to that of our time interval window so that it only looks for a maximum of one onset at the current window. Since we are iterating with a sliding window approach through our signal in our rhythm processor, we do not need to look for more than one peak at a current window. 

The height parameter is currently set to a certain number that I found to be the lowest value when playing a note based on multiple audios. However, in the future we might have to play around with this value when testing the system to try to maximize the performance of the rhythm processor based on different inputs. 

After that, my team and I mainly focused on writing the design document for the project. 

I think our progress might be a little behind overall, due to our team facing sick days and having heavy work from other classes. However, our other classes should hopefully get a little better on the coming weeks allowing us to catch up, and I think we will be fine based on timelines. 

For next week me and Aditya will be focusing on the integration of the rhythm and pitch processors.

Met with the team and prepared for presentation. I did practice for the design presentation by reading over all our content, ensuring it contained all required information and trying to internalize everything so that I could talk to the audience without having to barely read the slides (except for maybe reading specific numbers for data).

I looked into how we can detect the SNR of our signal by doing some research online.  It turns out SciPy used to have a function that calculated the SNR of a signal, but it was deprecated. However, the code for that function is still online and therefore we can use this to detect the SNR in our audio input validation system (here).

I also researched how the rhythm processor should work and how we might want to implement it (paper). After reading some articles, I figured it would be best to try to detect the onsets in a given audio signal to detect the start of a new note. Since we will be dealing with piano music, this makes sense since the onset of a new note will be instantaneous. 

I decided to use the find_peaks() function from SciPy. I will be iterating over the input signal in the time domain, by looking at sections of the signal of a certain length (as of right now we said 1/8th of a beat) and trying to find peaks there. If I find a peak in a section I will append a 1 to the found peak array and if not I will append a 0. That is the main idea of the algorithm, which will return an array of 1s and 0s which indicate if there is a peak in that section of the signal or not (so the value of index 0 would be the first section of the signal, the value at index 1 the second section of the signal, etc).

The code right now looks something like this, but it is still being tested as the find_peaks() function from SciPy is a little complex since it will require us to come to an agreement on what values to use for the parameters to make it work correctly. I will attach the code I wrote to write the algorithm and test it at the same time.

It can also be viewed here

The following courses were helpful in the design principles used in our project:

• 18-290: Fourier Transforms, Frequency Domain vs Time Domain Signals, Signal to Noise Ratio.
• 15-112: Object Oriented Programming.
• 17-437: Web Applications.

During this week I mainly focused on trying to find software that will help us developing the frequency processor. Last week I found out that the pitch function from MATLAB would be quite useful to have to test against our own frequency processor. The purpose of this function is when given an audio signal it would return to the user all the frequencies found for that signal in the right order. Therefore, we can compare this to our own frequency processor to determine how accurate our frequency processor is. 

The problem is that we will need to write this code in python since we need to build our web-app utilizing Django. At first, I tried utilizing a public library called CREPE. However, it was really hard to install this library, and I was not successful in doing so. I tried for 2 days to install it, but the library required me to install TensorFlow and it was causing me too much trouble, as well my teammates. 

I decided to look for other libraries and I found another one called Parselmouth. This library was easy to install and also has a Pitch function to determine the different pitches given an audio file. The software seemed to correctly detect the frequencies given a couple of files that we had. Therefore, we will be utilizing this for testing our frequency processor. 

These are the outputs of a C-note and a C-scale: 

I also have been working with my teammates in laying out all the content for the Design Review, as well as setting up the database for our django application.

I would say my progress is on-schedule. There have been some modifications to our Gantt chart and I will be focusing this upcoming week on researching how to do the rhythm processor, as well as preparing to give the presentation for the Design Review.

This week I helped on creating the proposal slides for the presentation that Aditya gave on Wednesday.

I was also supposed to think about the design of the data structure for the audio file in our code. I thought that we could have the following data structure for the audio file. The data structure would contain a field containing the length of the audio in minutes, the sampling rate of the audio in samples per second, the number of samples taken throughout the whole audio, and a field containing an object which would be containing the content of the Short Time Fourier Transform for our audio.

class audio {

length

samplingRate

samples

STTF{}

}

I also researched into some helpful tools that we could utilize for our frequency processor. I found out that MATLAB offers a function called “pitch()” that when given an audio file input it will give us an output containing the frequencies of the audio. This would be ideal for us to use it to detect what note frequencies are being played at a certain time. This is the output of the function for a C-scale audio for example 

and this is the output for a constant c-note

The only issue is that when the audio goes quiet we get inconsistent frequencies like for the first image at the beginning and at the end. Therefore, at moments of silence we would have to ignore the output of this algorithm and this might be a challenge we have to deal with in the future.

My progress is on schedule, and next week I will be focusing on designing the data structure for the Short Time Fourier Transform as well as helping my team in integrating the frequency processor in the back-end in Python.