Author: aruiz2

Progress

I realized that when reading the time signature form options what is being processed is the number as a decimal in python. Therefore, I had to make a switch statement and give the time signature of 6/8 a different value when selected than that of 3/4  since otherwise, they would have the same value. 

I also implemented a function in the backend that calculates the number of staves that we will need to draw dependent on the duration of the audio file. 

I developed the front-end code for the save pdf button that we added, and I helped Kumar with ideas of the implementation of the backend function that turns the Vexflow output to a PDF when the user clicks on the “save-pdf” button. 

Like we discussed in the Team Status report we are having issues with some audio files. Therefore, I tried to record the output of Twinkle Little Star with my phone and send that audio file to our system to process. It seems like some audio files are able to be handled but others give us errors. We are still not sure why this is occurring and therefore part of the focus of next week will be to figure out why. 

I also cleaned up code accross project by removing dead code and adding documentation to make sure we are all on the same page and have an easier time when trying to understand each other’s code. 

I finally edited the code that we had that drew the notes using Vexflow. Before, we looped all the time through the notes and when we reached the notes that had assigned a specific stave, drew them into that stave. This is inefficient because we loop through all the notes all the time, even though we only need the notes belonging to the stave we are drawing at that iteration of the loop. Therefore, I modified the way the notes were being processed and made a dictionary that matched staveIndex to the notes. Now, we just access the notes by getting the value assigned to a stave index and have reduced the complexity. The system did indeed get faster. 

We should be on track in terms of progress. 

For next week, I will be focusing on trying to figure out the issues with the audios, as well as helping my teammates out in whatever they need from the current tasks they are doing, since I know at least Kumar was facing issues implementing the backend of the save PDF button.

Testing

We will need to test the output of our integrator system. We have already started testing it by recording simple audios with the piano and feeding it into it. We will first need to test that the amount of notes in a stave is correct. We can test this by looking at the time signature and ensuring that the total length of the notes in a stave is less than the time signature, and that the note going into the next stave could not be added to the previous. We will also need to test that the amount of notes is correct. We can do this by listening to the audio and counting the amount of notes that we have heard and then count the number of untied notes that our integrator outputs. We should also test the accuracy of the pitches of the notes. We can do this by recording the notes pitch and comparing the notes pitch that our integrator outputs. Since we said we wanted it to be >= 90% pitch accuracy accurate, as long as it meets this threshold we should be good. We should also test smaller functions, like the function I created to compute the number of staves. This can be done by looking at an audio file and calculating how many staves we need for it and then comparing it to the output of our function.

This week I implemented the SNR-rejection system of our project. It will display a red alert in the front-end of the app if the audio is rejected due to it having too much noise. If the SNR-rejection system requirements are satisfied, the audio will be processed as usual (see here).

I also focused on testing our systems. I first created a bunch of piano files in Garageband containing different audios of monophonic piano files. The audios at first were pretty simple and short for us to be able to use these and see how well our systems work initially. Then, I made a couple of longer and more complex audios to be able to test our systems with something more realistic. 

I run these files on our systems and they seem to work fine on our rhythm and pitch processors but not in the integrator. At first one issue was the integrator was outputting all rests. I realized the issue was that we changed code to normalize the signal before putting it through the rhythm process and therefore I had to modify a parameter that sets the height required of a signal for something to be a peak to a much lower value since the signal output normalized ranges only from 0 to 1. 

After fixing that running the integrator is not being too accurate as of right now. It seems like some notes detections are innacurate in terms of pitch. We also discussed with Tom that the way we are currently integrating the systems might not be the best so we might have to re-implement the integration system. 

This week I will be focusing on the testing part and try to find what issues are going on with our integrator as well as possible fixes. I think we are on track.

First of all, I restructured the whole structure of our djangoapp application. The reason is we had created a lot of unncessary django applications for every subset of our app. So for example we had created a django application for our rhythm processor, one for our pitch processor etc. We do not need all these applications, just one for the music transcription application. Therefore, I deleted some folders and files, changed some import statements to handle the new structure,  and made it look cleaner.

I also created a django form called “AudioForm” which will allow the user to select the user-defined values in the front-end. These include the audio file, the type of clef, and the time signature.  This is how it looks…

I also made sure that the rhythm processor is able to handle audio files that contain more than a single channel. Since the rhythm processor is mainly looking for onset signals, I converted the several multiple-channel array into a 1D array. I iterated through each of the channels value at a specific instance and decided to just keep the value with the greatest absolute value and append it to the 1D array of the audio data that will be looked at to detect onsets. 

I would say my progress is on schedule.

Next week I plan to focus on writing code for the integration of the pitch and rhythm processor, as well as adding some styling to the front end of the website since it currently contains no CSS.