Qingyi’s Status Update for 02/22/2020

Accomplishments this week

1. Receive MIDI signals from the digital keyboard on Mac with Python 3.5.

• The final implementation used Pygame’s built-in MIDI module with python 3.5. Important documentation for Pygame’s MIDI module can be found here https://www.pygame.org/docs/ref/display.html.
• It’s not trivial to correctly install the Python 3 version Pygame on Mac. I’ve found this helpful documentation that provides step-by-step instructions on how to perform the task, and have included everything needed in our Repo’s requirement.txt.
• There are fewer available resources for getting live input from MIDI devices with Python online than I thought there should be. I decided to work with Pygame’s built-in MIDI module after taking the following aspects into account:
  • Advantages: Most of the midi libraries in Python have not been well-maintained since Python 2 Some of them only work with Python 2/Windows. Considering our limited accessibility to a Windows system and to ensure consistency over the whole project as well, I decided to stick with Python 3. Pygame is well-supported in Python 3 and works both on Windows/Mac, so I think this is a good fit for the purpose of this task.
  • Disadvantages: Pygame forces users to process events by polling, so the midi events fetched by Pygame are not in fact real-time.
• The final implementation used legacy code from  Mr. Kushal Bhabra here. I added packages needed for Mac to make it work since the original environment is on Windows.

2. MIDI pattern matching.

• MIDI matching works on perfectly matched audio slices last week; I tested the matching algorithm by slice a piece (size range from 30 notes – 900 notes) from the original midi file and match the sliced piece with the original midi file; if the offset is correctly aligned, then the two files can be said to have achieved a perfect match. The algorithm is proved to work on perfectly-matching slices. A sample test output is shown below:
• Since we cannot require players to play exacts the same (to play every note drawn; the interval between each note is not important since it’s not taken into account in the current version of the matching algorithm; the speed/rhythm of playing will not affect the matching result.) as what the original sheet music is. Thus, I’m moving on to a matching algorithm that does not require the two input slices to be exactly the same. I’ve written functions to randomly delete notes in a slice, and try to test out the matching algorithm on this dataset. However, now the Bag-of-Word algorithm seems to be not as promising as what it has shown on perfectly-matching slices. I’m trying out a variation of the Levenshtein distance which is mentioned here and suggested to be used in scenarios related to music melody.

Progress for schedule:

• On schedule

Deliverables I hope to accomplish next week:

• Start with Django; make a simple web interface with backend SQLite database to store all the sheet music and midi sequences we’ve collected.
• continue with the midi matching algorithm and hopefully can find a way to match slightly-different midi slices with an acceptable accuracy (19 out of 20 test slices should be successfully matched).