Posted on

Gene Chang’s Status Report for Feb 7

This week focused on acquiring, testing, and integrating the core hardware components for the LumiKeys system, while continuing to define the technical direction of the hardware architecture. Specifically, I focused on testing NeoPixel LEDs, force-sensitive resistors (FSRs), and their integration with the ESP32 microcontroller, as well as researching UDP-based communication between the server and ESP32 for user-input verification.

Hardware Design and Component Work

  • Researched and evaluated hardware components required for key-press detection and visual feedback, including NeoPixel LED strips and force-sensitive resistors.
  • Developed simple test programs to read analog values from FSR strips and characterize resistance changes under applied force.
  • Defined threshold values to distinguish between pressed and unpressed states for reliable key detection.
  • Investigated communication methods between the ESP32 and the server, focusing on UDP as a lightweight protocol for transmitting note and verification data.
  • Planned the next stage of LED control using the Adafruit NeoPixel library and preparation for soldering and breadboard integration.

Initial Technical Prototyping

  • Prototyped force-sensing resistor input using the ESP32 ADC to measure resistance changes and convert them into usable analog values.
  • Validated that force input can be reliably detected and distinguished through thresholding.
  • Evaluated power requirements and identified the need for a stable power supply and voltage regulation for driving LEDs alongside the ESP32.

Schedule

I am currently on schedule, as core hardware components have been obtained, basic sensing functionality has been validated, and integration planning is underway.

Deliverables for Next Week

  • Integrate the NeoPixel LED strip with the ESP32 and implement basic LED control.
  • Set up a stable power system, including an appropriate supply and a 3.3 V step-down converter where needed.
  • Begin combining FSR input detection with LED feedback to demonstrate a basic end-to-end hardware loop.
Posted on

Team Status Report for Feb 7

One of the most significant risks to the project is the complexity of integrating multiple software components together with the hardware layer. Because the system depends on several stages working seamlessly, any mismatch in interfaces or unexpected behavior during integration could delay progress. To manage this risk, we are intentionally designing the system to be highly modular, with clearly defined boundaries and data formats between components. Each module will be developed and validated independently through thorough test cases before being connected to the rest of the system. This approach allows us to verify correctness at the component level first, making integration more predictable and easier to debug.

Specifically a major risk for the project is the reliable nature of interpreting the sheet music under real-world conditions, in which given the factors of scan quality, lighting, and perspectives of the sheet music can affect accuracy of extracting necessary information and unfortunately if there are errors at this stage it would affect the final result of the LED. To manage this risk, the team will plan to design a software pipeline that will provide preprocessing, validation, and modular testing at each stage allowing inaccuracies to be noticeable and isolated before putting together other components.

An additional risk involves the responsiveness requirements that we set for the full system starting from the user input to LED output. The latency that will come from software processing, verification logic, and the software-hardware communication is something that is necessary to mitigate. To avoid these risks, we will measure the end-to-end latency across components, optimize paths, and avoid blocking operations between subsystems. If accuracy or latency measures are not met within the timeline, some backup plans include supporting alternative digital input formats or simplifying processing to maintain a functional user experience.

There is also a risk of hardware-related issues, such as FSR strip malfunctions or defects in the LED components, which are distinct from software bugs. To mitigate these risks, we plan to keep spare parts on hand and maintain fallback options. If FSR strips prove unreliable, we will explore alternative input methods such as IMU-based detection or MIDI input.

A minor change was made to the hardware sensing design during this stage of development. The original design assumed force-sensitive resistors (FSRs) would be used as the primary method for detecting key presses. After initial testing and evaluation, we decided to keep alternative sensing methods, such as IMU-based detection or MIDI input, as fallback options in case FSR reliability, durability, or sensitivity becomes a limitation in real use. This change was necessary to reduce technical risk early in development and ensure that the system can still meet functional requirements if the primary sensing method proves unreliable. The main costs associated with this change include purchasing additional components for testing and prototyping, as well as the extra time required to evaluate multiple sensing approaches. These costs will be mitigated by performing early validation to determine the most reliable sensing method and standardizing on a single solution once testing is complete.

There have been no updates to the schedule. Work is progressing as planned, with no new visual milestones to report at this time. The Gantt chart (Gantt) outlines the team’s current timeline.
Posted on

Chris Oh’s Status Report for Feb 7

This week I focused on defining the software architecture for the project as we prepared for the proposal presentation and aligned on the technical direction of the system. I concentrated on designing how the software pipeline will be structured and how it will interface cleanly with the rest of the system to support modular development and testing.

Planning & Software Architecture Design

  • Designed the end-to-end software pipeline and how it interfaces with other system components.
  • Helped structure and refine the proposal presentation to clearly communicate our technical approach and system design.
  • Defined major pipeline stages and the flow of data across them to ensure a modular and scalable structure.
  • Identified clear module boundaries, expected data formats, and interaction points so components can be implemented and tested independently before integration.
  • Focused on ensuring the architecture supports future testing, debugging, and incremental integration without bottlenecks.
  • Updated and refined the Gantt chart to better reflect task dependencies, realistic milestones, and anticipated implementation challenges.

Technical Design Work

  • Mapped out how each stage of the pipeline transforms data from input to output.
  • Documented design decisions around modularity, interface definitions, and testability.
  • Planned how individual modules will be validated through thorough test cases before being connected into the full system.
  • Finalized the software-side pipeline design, turning conceptual flow into a concrete implementation plan.

Schedule

I am currently on schedule, with the software architecture and pipeline design finalized and ready for implementation.

Deliverables for Next Week

  • Begin prototyping the conversion layer module.
  • Experiment with relevant Python libraries to support the conversion process.
  • Start translating the architectural design into working code and validating module behavior independently.
Posted on

Simon Lee’s Status Report for Feb 7

This week focused on planning the whole system with the team as we prepared our proposal presentation and continuously defining the technical direction of the software components of the project. Specifically, this week I focused on researching and learning more about how to implement the sheet music scanning mechanism and the OMR system as part of the LumiKeys architecture.

Planning & Team Design Work

  • Worked with Chris to design the end-to-end software pipeline and how integrates with the hardware layer with Gene, as well as work on refining proposal for the presentation.
  • Helped define major pipeline stages including:
    • Image preprocessing
    • Staff detection and segmentation
    • Musical symbol recognition
    • Temporal grouping and quantization
    • MusicXML generation
  • Discussed module boundaries and data flow to ensure each component can be developed and tested independently and how we can reach performance goals related to accuracy and latency for the scanning system

Initial Technical Prototyping

  • Performed exploratory preprocessing experiments on sheet music images using Python, OpenCV, and NumPy, including grayscale normalization, adaptive thresholding, and morphological filtering to evaluate staff and symbol separability
  • Analyzed sensitivity of preprocessing results to lighting variation and capture quality
  • Assembled a dataset of smartphone-scanned sheet music for controlled evaluation

Schedule

I am currently on schedule as core scanning architecture and requirements have been defined and implementation work has already started.

Deliverables for Next Week

  • Begin implementing staff detection and basic symbol recognition
  • Continue refining preprocessing techniques
  • Start building automated testing framework