Posted on

Chris Oh’s Status Report for April 4

This week I focused on integrating the software and hardware components of the system and stabilizing the end-to-end pipeline. The main effort involved connecting the playback and visualization software to the hardware layer, debugging integration issues, and resolving errors encountered during real-world execution. In parallel, I worked on fully integrating the OMR pipeline with the conversion layer and began addressing a Java-side error affecting OMR processing.

Software and Hardware Integration

  • Integrated the playback and visualization software with the hardware layer to enable end-to-end system execution.
  • Verified that processed musical events are correctly transmitted to the hardware during playback.
  • Encountered intermittent failures and runtime errors during integration, primarily due to timing, data formatting, and interface mismatches.
    Spent time debugging and resolving these issues to improve system stability and reliability.
  • Continued iterative testing to ensure that software-hardware communication behaves consistently during playback.

OMR and Conversion Layer Integration

  • Fully integrated the OMR pipeline with the conversion layer, enabling direct flow from sheet music input to processed musical events.
  • Verified that OMR outputs are correctly parsed and transformed into the system’s internal event representation.
  • Encountered a Java-related error within the OMR pipeline that affects certain inputs.
  • Began debugging the Java error and investigating its root cause to restore reliable OMR processing.

Schedule

I am currently on schedule. While integration exposed several failures and errors, this work is expected at this stage and is helping stabilize the system. Progress continues toward a fully integrated end-to-end pipeline.

Deliverables for Next Week

  • Resolve remaining software-hardware integration errors and improve system robustness.
  • Fix the Java error in the OMR pipeline and verify reliable sheet music processing.
  • Continue end-to-end testing from OMR input through playback and visualization.
  • Begin performance and stability testing under different tempos and input conditions.
Posted on

Team Status Report for April 4

Project Risks and Mitigation

As the project enters the validation phase, a key risk is that the system may not perform reliably under real user conditions, even if individual components work correctly. In particular, variability in sheet music inputs such as skewed images, shadows, or low-quality photos may affect the overall system output. This is being addressed by planning end-to-end validation tests using a range of realistic inputs to ensure the system behaves consistently beyond controlled demo cases.

Another important risk is maintaining synchronization between playback, visualization, and hardware output. While this was stabilized for the demo, validation will now confirm that timing remains consistent during full system operation. This is being managed by continuing to drive playback and visualization from the same event stream and testing the system over longer runs and different inputs.

A further risk is that errors from earlier stages may still impact the final user experience, even if they are detected. To mitigate this, the team is focusing on ensuring that failures are clearly communicated and that the system either produces usable output or fails in a predictable and understandable way.

On the hardware side, there is still a risk of inconsistent key detection from the FSR sensors. This will be evaluated during system-level validation by testing how well hardware input aligns with software-generated events, with filtering and calibration used to improve reliability.

Design Changes

This week did not introduce major architectural changes, but the focus shifted toward preparing the system for validation. Minor improvements were made to logging and output consistency so that system behavior can be more easily observed and evaluated during testing.

The system design continues to emphasize modularity, allowing individual components to remain independent while supporting full system validation. This ensures that issues identified during validation can be traced back to specific parts of the system.

Schedule Update

The project remains on schedule. The team has transitioned from demo preparation into the validation phase, which aligns with the planned development timeline. No major schedule changes are required.

Validation Plan

Validation will focus on evaluating the full system from a user perspective, ensuring that LumiKey works as intended for guided piano learning. The team plans to run several end-to-end tests that simulate real usage of the system.

The first set of tests will evaluate the complete user workflow, where a user uploads sheet music and the system processes it through the full pipeline to generate LED guidance and playback. The goal is to confirm that the system operates without failure and produces usable results.

The second set of tests will evaluate correctness of musical guidance. For selected test pieces with known outputs, the team will compare the generated note events and LED behavior against expected notes. This ensures that the system is guiding users correctly rather than simply producing output.

The third set of tests will focus on timing and synchronization, verifying that playback, visualization, and LED output remain aligned. This includes checking that the system responds within the required latency and that musical timing is preserved during playback.

The fourth set of tests will evaluate robustness under different input conditions, including clean PDFs, skewed images, and lower-quality photos. The goal is to ensure the system either produces correct results or fails in a controlled and understandable way.

Finally, hardware integration validation will confirm that the event data generated by the software is correctly interpreted by the ESP32, that LEDs display the correct notes, and that key press detection aligns with expected inputs.

The results of these validation tests will be used to determine whether the system meets the use-case requirements, particularly in terms of correctness, timing, and usability.

Demonstrated Progress

The team has successfully transitioned from demo readiness to a system capable of supporting full validation. The software pipeline remains stable and is able to process sheet music inputs, generate note events, and drive playback and visualization components.

In addition, the team has defined a clear validation strategy focused on real user behavior rather than individual components. This represents an important step toward ensuring that LumiKey not only functions technically but also meets its intended purpose as a guided music learning system.

Posted on

Simon Lee’s Status Report for April 4

This week I focused on transitioning from demo preparation into the verification and validation phase of the LumiKey project. Building on the previous work where the scan pipeline was stabilized for the demo, my goal this week was to define concrete testing procedures and evaluation metrics to measure whether the Scan Layer and full pipeline meet the design and use-case requirements.

Verification Planning for Scan Layer

For my subsystem, verification will focus on evaluating the performance and reliability of the Scan Layer, including preprocessing, OMR execution, and MusicXML cleaning. I plan to use a benchmark dataset of 15 to 25 sheet music inputs that include a mix of clean images, skewed photos, shadowed images, and PDFs to reflect realistic usage conditions.

The specific tests I plan to run include:

  • Running the full pipeline on each input and recording whether valid MusicXML is successfully produced.
  • Logging the exact stage where failures occur, such as preprocessing, OMR execution, or XML cleaning, to identify bottlenecks.
  • Measuring output consistency by running the same input multiple times and verifying that the resulting MusicXML and note events remain stable.
  • Comparing detected note counts and structural properties against expected values for selected test files.
  • Tracking error types such as missing OMR output, malformed XML, or parsing failures.

For analysis, I will compute the overall pipeline success rate across the dataset, with a target of at least 90 percent successful conversions. I will also analyze failure distributions by stage to determine whether issues are concentrated in preprocessing, OMR, or post-processing. Consistency checks will help ensure the pipeline behaves deterministically rather than producing variable outputs.

Validation Planning for Full System

At the system level, validation will focus on whether the full pipeline produces correct and usable outputs for the intended application of guided piano learning. This involves confirming that the generated note events accurately reflect the musical structure and can be used by the playback system.

The planned validation steps include:

  • Selecting 3 to 5 representative pieces with known correct outputs.
  • Comparing generated note events against trusted references such as manually verified MusicXML or expected MIDI sequences.
  • Evaluating correctness by checking pitch accuracy, note ordering, and basic timing relationships.
  • Verifying that the generated event stream integrates correctly with the playback and visualization system without errors or misalignment.
  • Observing whether the output is usable in practice, meaning that playback behavior matches the intended musical structure.

The main validation criteria will be whether the system produces correct and usable note events for known inputs and whether those outputs integrate cleanly into the rest of the system.

Next Steps for Testing

These tests have not yet been executed, so the next step is to implement evaluation scripts that automate running the pipeline on the dataset and collecting results. I will also improve logging so each test clearly reports success, failure, and failure reason.

After running the tests, I will analyze patterns in failure cases and identify areas where preprocessing, OMR integration, or XML cleaning need further refinement. This will allow targeted improvements rather than ad hoc debugging.

Schedule

The project remains on schedule. The system has transitioned from demo readiness into structured verification and validation, which aligns with the planned development timeline.

Deliverables for Next Week

Next week I will execute the planned verification tests on the benchmark dataset and analyze results against the defined success criteria. Based on these results, I will refine preprocessing and OMR handling to improve reliability. I will also continue validating system-level behavior by ensuring that the generated outputs integrate correctly with playback, visualization, and hardware components.