Author: shreyara

This week, I focused mostly on testing the beat map generator. I had two main goals for the beat map at the beginning of this project. The first was to have the beat map generator take less than the length of the audio file to analyze the audio file and output a beat map generator. To test that, I ended up making about 50 maps and then computing the time it took to test it. I put the graph on our team’s slides. I think that goal is fine.
The second one with the synchronization has been trickier. It’s pretty simple to get the number of beats down to make the game playable for our audience (one piece of feedback we got), but I’m still not super sure what’s happening with the synchronization. I have this idea to only compute the most apparent beats instead of all of them to improve synchronization, but that’s something I will be testing. Unfortunately, I have a few papers and exams due, so I will probably dedicate Monday to working on this problem. This all needs to be done by Thursday for the TechSpark demo. I also have a final Thursday evening so this will likely be done between Monday and Thursday

This week, I was focused primarily on testing for the presentation and report for next week. I am running a script right now for the timing portions of my model about determining the average time it takes to run for certain lengths of times (one of my user requirements).

The second user requirement is harder to talk about. I have a few metrics that I I am considering which is comparison to a commonly known beat tracker without regarding false positives or negatives. This is because I included the melody tracking and the commonly used package does not. I was also considering submitting graphs of the areas in an audio file where the algorithm would place a beat to see if we can draw some comparisons for that. I have the wav files created, so having the intensity analysis graphs created should be pretty fast.

Lastly, I think we can incorporate the user feedback and the results we received from play testing.

I do actually have a bug currently in my code which suggests its outputting the wrong time stamps. I am working on trying to fix it, but I am unsure if I can get to it this week. I may have to look more into it next week and focus on the testing analysis for this weekend.

Unfortunately, this week, I was pretty sick so I was absent from class.

However, I was able to make significant progress with the accuracy of the beat map creator. I ended up changing the energy calculation by making the window smaller than what it was before. I was calculating instantaneous energy in bins of 43 (recommended by the source I was using), but I ended up making the bin of length one instead. This also ended up being significantly faster. Currently, I am analyzing two frequency bands and splitting that into four buttons. I am analyzing the frequency bands separately and for a three and a half minute audio file, it takes about 112 seconds per frequency band to analyze. While that is slightly above my user requirement, tomorrow I plan to condense the algorithm into one loop so hopefully, it should take about 200 seconds total for a single audio file.

Also, as I mentioned last week, (George’s idea), I added a sound to indicate when my tracker found a beat. I have attached a few sound samples before. I have been creating a few additional beat maps as well for the testing phase of our game that we will be going through in a few weeks.

For the progress for the next week, I need to create a lot more files for testing as well as for verifying that my user requirements are met (or where they fall short). Furthermore, I need to create a main function to call everything instead of having to do it myself. This should only take a few hours because the code is pretty modular. Hopefully, I can have all of this done by Wednesday.

For scheduling, I should be on track. Everything is pretty much done, but I am working on some final touches.

NOTE: I’m having trouble uploading the files to the website because it says the maximum size of the files has been exceeded. I can show them later.

This week, I was mostly focused on the Interim Demo. Last week, I had the idea of using the STFT to separate the audio file into frequency bins to analyze through the intensity analysis. I thought this would be more efficient due to the variability of the number of bins. However, I was having some trouble implementing it.

I spoke to Professor Sullivan regarding the approach, and he mentioned that it would be better to use bandpass filters and then run an intensity analysis on that. I ended up implementing that with scipy and then running the intensity analysis on the filtered wav files.
On Wednesday, I had an odd bug where there would be no outputs during long stretches of time during the audio file. I was able to fix that by scaling the array, and then analyzing it. That issue is fixed.

Timing wise, while the algorithm takes about six minutes to analyze a three minute audio file, I should be able to get this down to three minutes as per the user requirements through streamlining the code. I will not be doing this for this week because this will likely be a finishing touch after tuning and testing the algorithm.

Currently, the beatmap seems to have an offset correlating to a song. George suggested adding a noise to the audio file whenever my algorithm detects a beat, so I should have that done by Wednesday so I can better understand whether there is an offset- or what exactly is happening.
I am fine on schedule. We have had to change the Gantt Chart which reflects our scheduling differences. The updated Gantt will be on this week’s team update.

 

This week, I was working on getting the gameplay code installed on my computer. That worked around Wednesday, so I was able to begin visualizing the beat maps.
For the second half of the week, I started looking at melody tracking algorithms. I figured out how to map the frequency bands to the timing with a STFT (graph shown below).

I’m working on extracting the melody itself and getting the time stamps. According to this paper, the algorithm is to determine the frequency of the melody line (orange line on the bottom of the graph) and then extract the time stamps from those frequencies. I’m currently looking at using an external package to get these frequencies, but I should have a weak melody tracker by Monday.

For scheduling, I should be fine. I’m working on the automatic part of the beat tracker and the melody tracker simultaneously, so I’m pretty much on schedule.

For this week, I was working on testing and verifying the accuracy of the beat map. I mentioned before that I didn’t particularly trust the accuracy of the beat map. While looking at the beat tracking accuracy on the game itself, there was a slight offset and then some issues with the accuracy of the beat tracker. I think this is primarily due to the energy calculation I had instituted.
Originally, I was working with the equation that the scalar multiplier was equal to c = (-.0025714*energy) + 1.5142857

This worked fine on the primarily instrumental audio file. However, with the popular pop songs, I was unable to use this algorithm because the shift in intensities is a lot more varied when looking at the beats. Through testing, I need to find a new variable function for c.
One issue we ran into was that it’s difficult to visualize the beatmaps without the game code itself. The code unfortunately does not work on Windows, so we are having some trouble testing it. That is also reflected on the new gantt chart in our team status report.
Unfortunately, this does put us a few days behind schedule and I may have to implement a simpler melody tracker to compensate, but that will be determined. I created a side testing platform with the Python standard audio processing data, but the visualizer is still more accurate. I can continue work on the scalar c function without it for now.
We are slightly behind schedule, but we should be able to complete the project in time.