Qingyi’s Status Update for 03/28/2020

Accomplishments this week

1. Advanced MIDI pattern matching tested live.

• Last week I tested the advanced MIDI pattern matching algorithm with live MIDI input on this Piano_Man_Easy.mid. The testing went through very well. The sheet music of Paino_Man_Easy is attached: Piano_Man_Easy. However, it does not mean that the matching algorithm works on every type of sheet music as I expected.
• First, in order to structuralize the program, I added the options.py file into the package,  which performs handling of all command-line arguments. Now user can run the readLiveMidi.py file using the syntax:

  python3 readLiveMidi.py –score Penguins_Game –window 20

  where the user input the name of the score in [score], and can optionally set the [window] argument, which specifies the number of live midi notes collected before running the advanced MIDI pattern-matching algorithm for one round. The default value of [window] is 10.

• Instead of running the MIDI pattern-matching algorithm every [window] notes as what I used to do, I changed the program to run the MIDI pattern-matching algorithm every 2 seconds, which, after experiments, is an acceptable time interval between which the tracking algorithm is run, because I find out that the original setup significantly slows down the program if the sheet music is rather complicated (when the sheet music writes lots of notes in one measure or when the user is playing with a high speed).
• Although testing went well with the file Piano_Man_Easy, when I tried to test the advanced MIDI pattern-matching algorithm on another file: Penguins_Game.mid. which contains significantly more chords and obviously notes than the song Piano_Man_Easy, it takes much more time to match the MIDI pattern every time, with the time needed ranging from 0.18s to 8s. The sheet music of Penguins_Game is attached Penguins_Game. Different [window] size results in significantly different processing time as well. However, it is indeed a tradeoff between the accuracy and the speed. As reported by the program, the time needed to prepare the info dictionary of the original midi file which is later needed for MIDI pattern-matching is 0.1s when [window] = 5, and exponentially grows to 35s when [window] = 10. However, if [window]=5, since there are lots of repetitive sections in the sheet music, the accuracy of the live matching is not very desirable; it is only capable of doing MIDI matching correctly if the user is playing exactly what is written on the sheet music. When [window]=10, the accuracy is desirable, and the matching algorithm can even perform correctly when the player rolls back several measures, repeat nearby measures, or play about 30% error notes. Yet, the matching algorithm runs rather slowly and results in glitches during the pattern-matching.
• In order to circumvent this case, I tried to come up with other possible optimizations to the program and implemented one version which gets rid of the necessity of pre-allocating space for the master info dictionary. However, it turns out that in order to make sure that the accuracy holds at certain levels, the new version still runs with similar time complexity. Before digging further into the optimization, I would like to wait until I migrate my code to AWS and run it there since the instance has much more storage and computation power than my computer does.

2. Setting up and migrating to AWS

• We get the AWS coupon code this week and have been working on migrating our code onto AWS so that my part and Lily’s part can be integrated together finally. We have successfully launched the AWS instance and have redeemed the coupon code.
• I’m currently working on the migration; I’m figuring out how to perform port forwarding so that my code running on AWS can read data from the midi keyboard connected to my local computer.

3. Integration: identifying the flip point

• integrated with Lily’s program locally: read the json output from Lily’s program, and now the MIDI matching program can successfully output the flip signal when the end of the page is detected.

Progress for schedule:

• On schedule

Deliverables I hope to accomplish next week:

• successfully run the current matching algorithm on AWS
• perform further optimization if necessary