This week I focused on validating and stabilizing the full end-to-end LumiKey software pipeline. Building on last week’s work where the Scan Layer was integrated with the OMR engine and connected to the existing Conversion Layer, the goal this week was to ensure that the entire flow – from image or PDF input to note event generation – works reliably and is ready for the interim demo.
End-to-End Pipeline Validation
A major focus this week was testing the complete pipeline: image/PDF input → preprocessing → OMR → MusicXML cleaning → conversion → note events. Since the Conversion Layer was already implemented by another team member, my work focused on verifying that the Scan Layer output integrates correctly and consistently produces valid inputs for downstream processing.
To support this, I created a small benchmark dataset of sheet music inputs, including clean images, skewed photos, shadowed images, and PDFs. This allowed me to test how well the system performs across realistic conditions. I also began evaluating outputs based on whether the pipeline succeeds, whether parsing errors occur, and whether the generated events are reasonable in terms of structure and count.
Evaluation and Debugging Improvements
I added a simple evaluation workflow that runs the pipeline across the benchmark set and reports results such as success/failure and basic output consistency. This helped identify weak points in preprocessing and OMR where certain inputs fail or produce inconsistent results.
In addition, I improved debugging visibility by ensuring that failures are tied to specific stages of the pipeline. Instead of failing silently, the system now makes it clearer whether an issue occurs during preprocessing, OMR execution, MusicXML cleaning, or parsing. This makes it much easier to diagnose problems and iterate quickly.
Additional Testing and Edge Case Handling
I added several targeted tests to improve robustness of the Scan Layer and pipeline. These include handling invalid inputs such as corrupted images, malformed PDFs, missing OMR outputs, and invalid MusicXML files. These tests ensure that the system fails in a predictable and controlled way rather than crashing or producing unusable results.
The goal of this work is to make the pipeline stable enough for demo conditions, where unexpected inputs or edge cases may occur.
Preparation for Hardware Integration
With the software pipeline now functioning end-to-end, I began preparing for integration with the hardware system. This includes defining and documenting the event format generated by the pipeline, which will be used to communicate note timing and pitch information to the ESP32.
This step is important to ensure that the software output aligns with the expectations of the embedded system, particularly for timing accuracy and LED control. It also sets up the next phase of development, where software-generated note events will drive real hardware behavior.
Schedule
The project remains on schedule and aligned with the interim demo timeline. According to the planned system pipeline and milestones , the current focus on integration and validation is consistent with reaching an MVP before the demo. The full software pipeline is now operational, and remaining work is focused on stability and hardware integration.
Deliverables for Next Week (Interim Demo Preparation)
Next week I will focus on final preparation for the interim demo. This includes refining the pipeline to ensure consistent behavior on demo inputs, improving error handling for a smoother user experience, and working with the team to connect the software output to the ESP32 hardware.
The goal is to demonstrate a complete working system where a user uploads sheet music and receives corresponding LED guidance on the keyboard. I will also continue refining preprocessing and validation steps based on any issues observed during integration testing.