This week I focused on integrating the Scan Layer preprocessing pipeline with the Optical Music Recognition (OMR) stage and building tools to validate that the preprocessing output produces reliable symbolic music data. My goal was to move the system from an isolated preprocessing pipeline to a functional end-to-end scan workflow that converts sheet music images into structured MusicXML data.
Integrated Audiveris OMR Execution
The main milestone this week was connecting the preprocessing pipeline directly to the Audiveris OMR engine. Previously the system stopped after producing normalized images, but it can now automatically pass the cleaned image output into Audiveris and generate MusicXML files. I implemented this integration using Python subprocess calls so the pipeline can invoke Audiveris programmatically and capture the resulting MusicXML output.
Once this was working, I tested the system on multiple sheet music samples to verify that the preprocessing improvements from last week were producing images that Audiveris could reliably interpret. I also validated the generated MusicXML by importing the results into MuseScore, which made it easier to visually compare the recognized notes against the original sheet music. This proof-of-concept confirmed that the preprocessing stage is producing OMR-compatible images and that the overall scan pipeline is technically feasible.
Implemented MusicXML Validation and Cleaning
After OMR execution, I began implementing a validation and cleanup stage for the generated MusicXML. OMR output can occasionally include inconsistent metadata, missing attributes, or formatting irregularities that can break downstream parsing. To address this, I used the music21 library to parse the generated MusicXML files and enforce a consistent structure.
This stage verifies that measures are correctly defined, removes unnecessary metadata, and ensures that note and duration information can be safely extracted by the Conversion Layer. The goal is to guarantee that the output from the Scan Layer is not only readable but also structurally reliable for the rest of the LumiKey pipeline.
Built Visualization Tools for Scan Layer Processing
To better understand how preprocessing affects OMR performance, I developed visualization scripts that highlight the structural features extracted from sheet music images. These tools overlay detected staff lines, barlines, and candidate noteheads on top of the processed image, making it easier to confirm that the system is correctly identifying musical structure before OMR is executed.
These visualizations also helped refine several preprocessing parameters, such as thresholding behavior and morphological filters used to isolate staff lines. In addition to helping with debugging, the visualization outputs will also be useful for illustrating the Scan Layer architecture in our design report.
Status Report 4 Requirements: Part B
Cultural factors were considered in the design of LumiKey because music education is strongly shaped by cultural traditions, learning practices, and accessibility within different communities. Many cultures place significant value on learning traditional instruments such as the piano, yet access to formal instruction and specialized digital instruments can vary widely depending on economic and regional circumstances. By designing LumiKey as a retrofit system that works with existing acoustic or digital pianos, the project respects the cultural importance of traditional instruments while lowering barriers to entry for learners. The system also avoids requiring proprietary hardware or software ecosystems, allowing it to be adapted for use in diverse educational settings, including home practice, community music programs, and classrooms across different cultural contexts.
Schedule
I am currently on schedule. The Scan Layer is now capable of running the full pipeline from image input through preprocessing and OMR execution. The remaining work in this stage primarily involves refining MusicXML cleanup and validating the pipeline on a larger dataset of sheet music images.
Deliverables for Next Week
Next week I plan to finalize the MusicXML validation logic and begin implementing the Conversion Layer that transforms symbolic music data into the time-indexed event representation used by the embedded system. I will also run the full scan pipeline on a larger dataset of sheet music images to evaluate pitch extraction accuracy and ensure the system can consistently meet our ≥90% correctness target. Additionally, I will work with the rest of the team to begin connecting the software pipeline to the ESP32 communication layer so recognized notes can be transmitted to the hardware for LED playback.