Completed this Week
Due to an IMU and ESP malfunction and a lack of backup components, we temporarily replaced the IMU with a ball tilt sensor. During this period, we successfully implemented several key features:
- Guitar sound mode with customizable note delay timing (which will eventually be calculated from IMU velocity data based on strum speed)
- Sound overlap toggle that functions like a piano sustain pedal. When enabled, previous chords continue playing while new strums are added; when disabled, each new strum immediately stops the previous chord
- Force-sensitive resistor integration requiring thumb-and-index-finger pressure to produce sound, mimicking the action of a real guitar pick
- Dual “3” gesture recognition based on user feedback from last week’s testing
- Glove integration so that the breadboard is on the glove rather than having to hold it separately
Here is a video of our current implementation before glove integration.
Collaboration with Music and AI course
We continue testing our instrument with musicians from the Music and AI course, and always use a feedback form to compile all feedback. We also discussed making an interface to help people learn songs with our instrument, and received feedback from a music education major for suggestions on what it should teach, and what it should not restrict.
Significant Risks
Our primary risk involves velocity calculation. Since the IMU provides only acceleration data, deriving velocity requires integration, which is non-trivial and susceptible to drift errors. This could compromise our delay implementation. Our contingency plan is to provide a manual delay selection bar with preset levels rather than dynamic velocity-based timing (which is our current implementation).
Changes Made to Existing Design
The ball tilt sensor is a temporary substitute for the interim demo. We will revert to the IMU once replacement components arrive.
Updated Schedule
We updated our Gantt chart to match with our progress and future steps that we need to complete the project.
Tests
Gesture Recognition Accuracy:
Test Method: record 50 repetitions of each gesture and measure classification accuracy
Success Criteria: >= 95% accuracy for each gesture type (per design requirements)
Analysis Approach: Generate confusion matrics to identify misclassification patterns
IMU Velocity Calculation
Test Method: Mark two points 30cm apart (typical strum distance). Perform controlled strums at three speed categories (slow/medium/fast), with 10 repetitions each. Time each strum using a stop watch or smartphone slow-motion vide. Calculate the actual average velocity using v_actual = distance/time. Simultaneously, log IMu acceleration data and integrate to calculate velocity over the same time period.
Success Criteria: Velocity estimates within ~15% of actula values for all speed ranges; consistent performance across trials with standard deviation < 10% within each sped category; minimal velocity drift over 30-second continuous operation
Analysis Approach: Fro each trial, calcualte percent error (|v_IMU – v_actual| / v_actual * 100%). Compare mean error and standard deviation for each speed category.
Force Resistor Threshold
Test Method: Measure activatino force required and response time; test with 10 users of varying hand sizes.
Success Criteria: Consistent activation between 2-5N of pressure; < 50 ms latency
Analysis Approach: Plot force-resistance curves; identify optimal threshold value that minimizes false triggers while maintaining responsiveness.
Audio Playback Timing
Test Method: Measure actual delay between gesture detection and sound output
Success Criteria: Latency <100ms (imperceptible to users)
Analysis Approach: Statistical analysis of latency distribution across 100 trials
Validation Testing
Musical Playability
Test Mthod: Have 5-10 users (mix of musicians and non-musicians) attempt to play simple songs (e.g. Twinkle Twinkle, basic chord progressions)
Success Criteria: Users can successfully play recognizable melodies within 5 minutes of instruction
Analysis: Record success rates, time-to-competency, and user satisfaction surveys (1-5 Likert scale); collect qualitative feedback on naturalness of interaction
System Reliability
Test Method: 2-hour continuous operation test with gesture cycling every 30 seconds
Success Criteria: No system crashes, <5% gesture recognition degradation over time
Analysis Approach: Log all errors, memory usage, and performance metrics; identify failure modes
Integration Testing
Test Method: Full system operation testing all subsystems simultaneously (gesture recognition + IMU + force sensor + audio + overlap control)
Success Criteria: All features function correctly without interference; no conflicts between subsystems
Analysis: Systematic testing of all feature combinations; identify edge cases and race conditions