This week I focused on executing the verification and validation plans defined previously and analyzing the results to evaluate the reliability of the Scan Layer and full LumiKey pipeline. Building on the testing framework outlined last week, my goal was to run the planned tests, identify failure patterns, and refine the system based on observed issues. In addition, I began preparing for the final presentation next week by ensuring the system is stable and clearly demonstrable.
Verification Results for Scan Layer
I ran the full pipeline on the benchmark dataset of sheet music inputs, which included clean images, skewed photos, shadowed images, and PDFs. For each input, I recorded whether valid MusicXML was produced and tracked the stage at which failures occurred.
The main results observed were:
- The pipeline achieved a high success rate on clean images and PDFs, but performance dropped on more challenging inputs such as skewed or poorly lit photos.
- Most failures occurred during the OMR stage, particularly when preprocessing did not fully normalize lighting or alignment.
- Preprocessing and XML cleaning stages were generally stable, with fewer failures compared to OMR.
- Output consistency was strong for repeated runs on the same input, indicating that the pipeline behaves deterministically.
Based on these results, I identified preprocessing improvements as a key area for increasing overall reliability, particularly for handling lighting variation and skew. The measured success rate is close to the target of 90 percent for clean and moderate inputs, but still needs improvement for more difficult cases.
Validation Results for Full System
At the system level, I evaluated whether the pipeline produces usable outputs for guided piano playback. For selected test pieces with known expected outputs, I compared generated note events against trusted references.
The results showed that:
- For successful OMR outputs, the generated note events were generally correct in pitch and ordering.
- Playback and visualization components were able to use the generated event stream without errors.
- In cases where OMR output was incomplete or incorrect, the resulting note events were also affected.
These results confirm that the system behaves correctly when the Scan Layer succeeds, and that improving early-stage reliability is important for overall system performance.
Refinements Based on Testing
Based on the testing results, I made several refinements to improve system stability:
- Adjusted preprocessing parameters to better handle skewed and unevenly lit images.
- Improved error handling to detect and report incomplete OMR outputs.
Strengthened logging to better track failure cases and support debugging.
These changes improved performance on several previously failing inputs and made the pipeline more stable overall.
Final Presentation Preparation
In preparation for the final presentation next week, I focused on ensuring that the system performs reliably on selected demo inputs. I identified a set of stable sheet music examples that consistently produce correct outputs and verified that they work end to end with playback and visualization.
I also reviewed the pipeline flow and key components so that I can clearly explain how the Scan Layer integrates with the rest of the system. This includes being able to describe preprocessing, OMR, and how outputs are used for playback and hardware interaction.
Schedule
The project remains on schedule. The system has progressed from testing into refinement and presentation readiness, which aligns with the final project timeline.
Deliverables for Next Week
Next week I will finalize preparation for the presentation and support the live demos on Monday and Wednesday. I will also address any last-minute issues that arise during testing and ensure the pipeline remains stable under demo conditions.
Additional Reflection
Throughout the development and testing of the Scan Layer, I learned several new tools and concepts that were necessary to complete the project. One major area was working with the Audiveris OMR engine, including how to execute it programmatically, handle different output formats, and debug cases where it failed to produce usable results. I also developed a deeper understanding of MusicXML and how it is structured and validated before being used by the rest of the system.
Another important area was image preprocessing using OpenCV. I learned how to apply techniques such as adaptive thresholding, illumination normalization, skew correction, and morphological filtering to improve the quality of sheet music inputs. These techniques were essential for making the OMR stage more reliable.
I also gained experience working with PDF processing tools such as pypdfium2 to support PDF inputs and integrate them into the same pipeline as images. In addition, I improved my understanding of building and debugging multi-stage pipelines, including how to design tests, track failures by stage, and analyze system performance using structured metrics.
To learn these tools and concepts, I used a combination of strategies. I referred to official documentation for libraries such as OpenCV and music21 to understand how functions work and how they should be used. I also used online resources such as Stack Overflow and GitHub discussions to troubleshoot specific issues. In many cases, I wrote small test scripts to experiment with different approaches and observe their behavior. I also watched short tutorials to quickly understand new concepts before applying them directly to the project.
Overall, these learning strategies allowed me to quickly acquire and apply new technical skills while working across different parts of the system.