Book-Keeping
I made a list of the final elements we have to wrap up in the last two weeks of the semester, and met with Shravya and Peter to assign tasks.
New Software Functionalities
I adjusted the software so that the user could open an existing composition without opening an external window. In order to do so, I implemented some logic where the program opens the next available file and loops back around to the first if it reaches the end of the available songs in the folder. Again, this obviously has some drawbacks in that the user may have to iterate through many compositions before reaching the end of the list, but it is accessible and simple, two of the most important foundations of our UI.
Another quick fix I made was to make the buttons different colors, at Peter’s suggestion, in order to make it easier for the users to differentiate between the different buttons and memorize different commands easier. Peter also mentioned that it would be good if there was a delay between calibration and the eye-tracking beginning right away so the user could orient themselves to the UI, so I also added in a short delay for that.
I also created a separate file that users could easily adjust sampling delay, gain and number of iterations for confirmation for the eye-tracking, in order for the user to be able to adjust those things as best suits them.
I had previously identified a bug in which only the first page of sheet music would appear on the secondary UI, even if the cursor was on another page. I thus added two more buttons on the screen so the user could move back and forth between the pages of the sheet music. I verified that this worked with sample compositions from this repository: [1]. (That were greater than one page).
I also made it so that the rest and stack buttons would be highlighted when they were selected but before they were performed (so the user is aware of them).
Finally, since I have been having so much trouble with highlighting the cursor location on sheet music, mainly due to the variable locations of any one cursor location depending on any sharps/flats and preceding notes, etc., I decided to show the user the current cursor position on the main UI as a number (e.g. “Cursor at note 1.”). This was not our original plan, but I still believe it to be a viable solution to ensure the user knows where in the piece they are.
Debugging
I fixed a small bug where the C sharp note was highlighted when the high-C note was confirmed and another small bug in which the number of notes did not reset to 0 when a new file was opened.
Then, I did some stress testing to confirm that the logic used to build rests and chords up to three notes was completely sound. While testing the eye-tracking, I had been running into some situations in which it appeared that the logic broke, but I could not figure out why. I used a different Python script that would run the command responses on key press (rather than eye-press) to easily test this functionality. Here were the edge case bugs I identified and fixed:
- When a rest is placed after a single eighth note, the eighth note is extended to a quarter note. I discovered with pretty certain confidence that this is actually a bug in the open-source sheet-music generator that we are using [1], because when I would play the composition in GarageBand, the eighth notes would play where expected even though the sheet music generator was interpreting them as quarter notes. I further verified by using another online sheet music generator, which also produced the expected value: https://melobytes.com/en/app/midi2sheet. Since I have already been struggling to modify this open-source program to highlight the cursor location and this is a very specific edge case, I decided it would be a better use of my time to not try to fix that bug within the program, and instead focus on our code. I have left a note in the README identifying the bug to users.
- Removing chords did not work successfully. I fixed this bug and verified it did not create a bug for single-note removal. Also, I verified that chords could be removed from anywhere in the piece, not just the end.
- There were some logic errors with chords meant to have rests before them, which I fixed. I also optimized the three note chord logic to avoid bugs, although I hadn’t identified any yet.
Next Week
After double-checking that the eye-coordinates, of which there have been some discrepancies, I plan to start performing the latency, accessibility and accuracy tests on the software. As a group, we will work on the poster and video next week so my primary goal is to finish software testing ASAP for those deliverables, but I will also work on other elements of them as needed. Additionally, I have to collaborate with Shravya once her UART code is working to integrate our two systems, but I have already set up some code for that so I do not anticipate that taking too long. For the final report, I will work on Use-Case & Design Requirements, Design Trade Studies (UI), System Implementation (UI), Testing (Software), and Related Works sections.
References
[1] BYVoid. (2013, May 9) MidiToSheetMusic. GitHub. https://github.com/BYVoid/MidiToSheetMusic