This week I focused on expanding and stabilizing the Scan Layer so it works as a reliable part of the full LumiKey software pipeline. Building on last week’s work integrating the preprocessing system with the Audiveris OMR engine, the goal this week was to improve preprocessing, strengthen OMR execution, and make the system easier for teammates to run and test.
Expanded Scan Preprocessing Pipeline
The preprocessing module was expanded into a full normalization pipeline capable of handling both images and PDFs. When a user uploads a PDF, the pages are first rasterized using the pypdfium2 library so they can go through the same preprocessing steps as images.
The preprocessing pipeline now includes illumination normalization to reduce lighting artifacts, perspective correction for photos taken at an angle, skew estimation with deskew correction, adaptive thresholding tuned using staff spacing, and morphological filtering to remove small artifacts. These steps help produce cleaner images that are easier for the OMR engine to interpret. I also reorganized the code into smaller helper functions so that image processing steps are easier to maintain and test.
Improved OMR Integration
The OMR module was updated to run the Audiveris engine directly through the command line using Python’s subprocess interface. The system now automatically detects the Audiveris binary using environment variables, system paths, or common installation locations on macOS.
Additional handling was added for locating the MusicXML output generated by Audiveris, including support for both .musicxml and compressed .mxl files. When compressed output is produced, the system automatically extracts the MusicXML so it can continue through the pipeline.
MusicXML Validation Improvements
The MusicXML cleaning stage was improved to make the pipeline more reliable. The cleaner now performs basic validation checks such as newline normalization, empty-file checks, and XML parsing verification. If the MusicXML file is malformed or empty, the system produces clear error messages instead of failing silently. This helps ensure the Scan Layer always outputs valid data before it reaches the next stage of the pipeline.
Testing and Pipeline Validation
A major focus this week was improving testing coverage for the scan pipeline. I added several test files to validate preprocessing, OMR integration, MusicXML cleaning, and the full scan pipeline. These tests check that the system behaves correctly when processing different types of inputs and when errors occur.
The current test suite contains 27 passing tests. I also added a real sheet music example (twinkletwinkle.pdf) as a test asset so the full pipeline can be validated using realistic input.
Project Setup Improvements
I also made improvements to the project setup process so other team members can run the system more easily. I added a bootstrap script that automatically creates a Python virtual environment, installs dependencies, and detects or installs Audiveris on macOS. The UI launch script now calls this setup step automatically before starting the application. I also updated the README with clearer setup instructions so teammates can run the project with a simple command.
Schedule
The project remains on schedule. The Scan Layer can now process both image and PDF inputs, run the Audiveris OMR engine, and produce validated MusicXML outputs. The system is also supported by automated tests, which will help maintain stability as new features are added.
Deliverables for Next Week
Next week I will focus on validating the full pipeline now that the Scan Layer is integrated with the existing Conversion Layer. I plan to build a small benchmark set of about 15–25 sheet music files, including clean photos, skewed photos, shadowed images, and PDFs. This dataset will be used to evaluate how reliably the end-to-end pipeline works.
I will also add an evaluation script that runs the pipeline on the benchmark set and reports results such as success or failure, parse errors, and basic note matching where expected outputs are available. In addition, I plan to improve error reporting so the application clearly identifies which stage of the pipeline fails if a scan does not succeed.
Finally, I will begin preparing for hardware integration by documenting the event format used to send note data to the ESP32 and adding a test to ensure the event serialization remains consistent for hardware communication.