Jiameng’s Status Update for 03/14/2020

Accomplishments this week

• Tested the cadenCV and made some modifications to the library code since originally the module does not recognize repetitions and end of the page, it only shows all the notes, sharp, flat and rests
• Currently, the code works quite well, for simple scores with no tenuto and grace note, also, the notes could not span too far above and below from each other, accuracy could reach an average of 96.92%, while for more complicated scores, the accuracy drops to 95.03%
• The procedure of adding any music score into the database for recognition and sending signals currently work like this, and take one score as an example (Piano man):
  1. Download a pdf score from MuseScore
     • Link for the score Piano Man
  2. Convert the pdf score to png files, take the first page as an example
     • 
  3. Run each png file through the code, and within code:
     • Need to set the threshold to control how many black pixels in a line would be considered staff to get the correct staffs before processing the score
     • 
     • The red boxes will mark the areas that are recognized as staffs for later recognition as shown in the image below
     • 
     • After getting the correct parts of the staffs, for each staff, the algorithm will find its rests, clefs, notes, and repeats if any exists
     • 
     • Also, the algorithm will output a text file with lines of the format: “note, position_x, position_y” or “start, position_x, position_y” or “end 1”, to show the position of notes, repeat signs and end of which page in order for processing afterward
     • The positions are x, y positions on each of the staff and not positions on the page
  4. Run the text file with another program, that will utilize these lines of information and extract the number of notes before a repeat signal or the number of notes before the end of page and output these to the MIDI matching module. Within the program, the logic works as below:
     1. Read all the title from database and get all the folders which contain output from the previous program
        • 
        • Each of the folders contains output for one score
     2. Within each of the folder, there are numbered folders where the number represents the corresponding page for that score
        • 
     3. And in each of the folder, there is a detected_staffs file which shows the page with red boxes for each staff, an output midi file, a positions file which shows all the positions for the notes, and primitives which shows the notes on each staff
        • 
     4. The algorithm runs through these files and counts the total number of notes within one numbered folder to mark the end of a page, and within position file, there are also lines that shows “end n” which means end of the nth staff, and this is used to calculate the position of repeat signals

Progress for schedule

• On schedule
• Changed the schedule
  • Originally, plan to only recognize the note this week and work on getting the basic features to work for web application
  • Currently, have the recognition work for both music notes and repetition and page-flipping and can output JSON data with different signals to the MIDI matching module
  • Changed the schedule because not sure how long note recognition will take and have some experience with a web application so that time is predictable and decided to finish recognition first

Deliverables I hope to accomplish next week

• Get around 15 more scores into the database to reach a total of 20 working scores
• Get the page redirect feature of web application working and have the HTML of all the sites display correctly