This week I focused entirely on designing and implementing the Scan Layer preprocessing pipeline for LumiKey. My goal was to build a robust system that converts raw phone photos of sheet music into normalized, OMR-ready images suitable for Audiveris. I built, refined, tested, and structured a modular preprocessing system that is tuned for phone photo conditions as well as PDF uploads.
Rebuilt and Stabilized the Full Normalization Pipeline
Instead of keeping preprocessing as a collection of loosely connected image operations, I reorganized it into a clearer, more structured pipeline. I separated configuration, image processing steps, quality checks, and utility functions into different files so the system is easier to understand and modify. The main preprocessing function now returns a structured result that includes whether the scan passed, relevant metrics, and specific failure reasons, rather than just showing intermediate debug images. This makes the behavior much more transparent and easier to improve.
I also improved the page detection step by tightening how contours are filtered and adding fallback logic when the page can’t be confidently detected. After perspective correction, I normalize the image resolution and ensure the page is upright before continuing. These changes make the preprocessing output more consistent and reliable across different phone photos.
Replaced Basic Contrast Enhancement with Illumination
Last week I was using CLAHE to improve contrast, but this week I replaced that with a better lighting correction method that works more reliably for phone photos. Instead of just boosting contrast, I now estimate the background lighting and normalize the image to reduce shadows and uneven brightness. This made a noticeable difference when testing on real photos taken under indoor lighting.
I also improved the black-and-white conversion step. I’m now using adaptive thresholding, and if the result looks clearly wrong (for example, too much black or too much white), the system automatically falls back to a different thresholding method. This helps prevent cases where the sheet music becomes unreadable due to poor lighting.
Finally, I refined the deskew step so that the image is only rotated when the detected angle is reasonable. Previously, large or incorrect skew estimates could make the image worse. Now, the system avoids over-correcting and instead flags the image if it’s too distorted.
Implemented Formal Quality Gate and Testing Framework
The biggest improvement this week was adding a proper quality check stage. Before, I was mainly checking whether the image was blurry, whereas now the system checks multiple things: whether the image is too blurry, whether there’s too much glare, whether the black-and-white conversion looks reasonable, and whether the page is still slightly rotated after correction. If something looks wrong, the preprocessing step doesn’t just fail silently, but rather provides reasons for what went wrong. This makes it much easier to understand and improve weak spots, and it also lets us track whether we’re meeting the ≥90% usable scan goal.
I also added automated tests using pytest to make sure key parts of the pipeline behave consistently, such as blur detection and skew correction. On top of that, I built a simple script that runs preprocessing over an entire folder of phone photos, saves the cleaned outputs, and logs which ones failed and why. Instead of manually eyeballing results, I can now evaluate performance in a more systematic and measurable way.
Schedule
I am on schedule. The preprocess stage has been evolving into a structured, modular, and testable system as thought out by the team from a partially implemented prototype. Remaining work primarily involves larger-scale testing and integrating with the OMR stage.
Deliverables for Next Week
Next week, I plan to connect the preprocessing pipeline directly to Audiveris so the entire scan process can run automatically from image input to MusicXML output. I’ll also begin implementing the MusicXML validation and cleaning stage to make sure the OMR output is structurally sound before it moves into the conversion layer. In addition, I want to run preprocessing on a larger batch of real phone photos to measure our actual usable scan rate and fine-tune parameters as needed to confidently meet the ≥90% target.