This week our team met with Professor Roger Dannenberg to get his insight into our ideas and what would be more plausible for our project. He told us some more details about the query by humming project and how its poor performance resulted in it not being a product that could go on the market. The act of breaking down music into separate lines is actually quite difficult since it’s hard to determine instrument lines and such. He mentioned F0 estimation which tries to find the primary note at each instance, which is essentially melody tracking, but also has mixed results. His suggestions have shifted our path more towards looking into using similarity matrices to match the songs. Similarity matrices essentially plot one song against another, and searching for diagonals locates the places where the two songs match up.
The similarity matrix idea also helps to play into the data visualization aspect of our project. Matrices are inherently visual so if we end up using this technique then the work that has been done by the matching algorithm through the matrix can be shown.
A point brought up during our weekly meeting with Professor Savvides and the TAs was that including a data visualization portion would help with debugging as well. I read this article which addresses some of the existing software that people use for similar things, such as Tensorflow Graph Visualizer. We will probably end up using those softwares, or at the very least employing techniques from them.
Unfortunately I have technically fallen behind schedule. I did not anticipate how much more research and planning we would have to do before moving on to other tasks, such as designing the data format. However, I have furthered my understanding of the problem at hand sooner rather than later, so I have a better idea of what will work now rather than having to turn back to try a different solution too late.
For next week, I would like to have a more concrete plan of how we are going to go about implementing both the audio analysis and the matching algorithm and explore how that will interact with the data visualization aspect of the project.