I have spend this week expanding on the timing algorithm. I have implemented a dynamic time warping system to correlate the sheet music to the audio data. This is a system that measures the similarities between two time series data sets and finds the similarities regardless of speed. This ensures that no matter the speed of the two sets of data it can still find a match. Attached is an example where the algorithm matched notes even though the second one is delayed by a random magnitude for each note. The left is the sheet music and the right is the delayed music.
This has just been with singers music so far as I am still figuring out how to handle pianos chords, but so far I have done the same process while just handling one note at a time.
Then to find if they are in sync I compared the sheet music between the two and found where they had shared note onset times. This is then used to compare each piece of audio data to find delay.
Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
My progress is on schedule.
What deliverables do you hope to complete in the next week?
For next week I hope to get some processed audio data from Ben to make sure the system still works and if there are any tweaks I need to make with that.