Posted on

Team Status Update 10/17

Model Updates:
Models both have results on songs, best for predictive is 59% and best for generation is 77.8%.

Risks are that 80% isnt reachable, so we either decide on changing target, or accept a lower threshold as more reasonable given typically playing habits.

Generation quality hasnt been tested either, just the next note prediction, so this has to be done manually by testers

Jai: Part B:
Culturally, there is an argument about not having generative AI inside of the arts. Just because generative tools like Suno already exist doesn’t mean making more is acceptable. However this is meant to be a tool which has human input which leads the song, rather than creating something purely for commercial gain, taking the human out of the music. The goal is for someone to play the guitar and be able to get a fuller experience as if they were in a band. It does not replace the band, and it doesn’t lead the experience. Rather is accentuates a guitar player’s own capabilities via the accompaniments.

Posted on

Jai Status Report 10/17

For prediction model statistics

chords of history top-1 top-3 top-5 top-10 next-chord-change top-1
1 0.375 0.634 0.750 0.884 0.328
4 0.453 0.683 0.782 0.896 0.412
8 0.497 0.712 0.801 0.903 0.454
16 0.527 0.732 0.814 0.909 0.484
32 0.558 0.752 0.827 0.914 0.517
64 0.583 0.768 0.838 0.918 0.539
128 0.590 0.773 0.841 0.920 0.542
label classes top-1 repeat-last margin
full 49 0.590 0.289 +0.301
major/minor only 25 0.620 0.320 +0.300
root only 13 0.656 0.362 +0.294

The above chart is about removing complexities from the chosen chord and counting those as the same chord or separating them.

error type share of errors
predicted the previous chord again 33.4%
wrong root, unrelated 23.6%
right root, wrong quality (e.g. 7 vs major) 15.5%
root a whole tone away 10.2%
fifth or fourth away, same quality 9.1%
no-chord involved 5.0%
relative major/minor 3.3%

It is very possible that the 80% goal is unreasonable. In many songs at any given point, a player is strumming the same chord over and over again so it makes sense that this is a prediction error. Additionally adding more training data didnt help after ~60k.

Tested different model sizes for generations

size params nats/note (lower is better) top-1 clean top-5 clean top-1 with +/-30 ms jitter top-1 jitter + 80 ms sort p95 decode, ms/token train hours per epoch
XS 10M 1.665 0.748 0.923 0.490 0.730 0.90 3.6
S 26M 1.592 0.758 0.928 0.499 0.741 1.42 8
M 53M 1.521 0.772 0.934 0.512 0.754 1.85 14
L 114M 1.479 0.778 0.936 0.521 0.762 3.25 25
XL 207M 1.467 0.778 0.935 0.525 0.761 3.94 43
XXL 404M not trained – – – – 10.13 about 90

This metric is given the previous information, how accurate is the generated token given the prediction.

The largest model ran out of memory on the 32gb gpu so I couldnt train it

Behind on training the XXL model but most likely I’ll skip it. It’ll be slower based on the trend, not helping accuracy.

Next week i’ll do pretraining for M or L model. Measure latency on laptop(or just keep it and test on the lab device and try to sample input and output on it.

Do more chord evaluation and latency, including when paired with the predictive model

Posted on

Tim’s Status Report (10/3)

MIDI Guitar Hardware Update

Donor guitar

  • Fender Squier Telecaster (25.5″ scale, 21 frets)
  • Chosen for its bolt-on neck, roomy control cavity, and low cost
  • An initial check showed 18″ from the nut to fret 21, which matches the theoretical 25.5″ scale layout

Fret sensing plan

  • A custom flex PCB overlay sits on the frets, with 126 pads (6 strings × 21 frets)
  • Each pad has its own trace, and the grounded string pulls a pad low when pressed
  • This lets us detect chords with several strings on the same fret

Measurements we’re collecting for the PCB layout

  • Nut-to-fret-crown distances, checked against the calculated fret positions
  • Fret crown width, height, and length
  • Fretboard width at the nut, fret 12, and fret 21, plus the fretboard radius
  • Each string’s lateral position at frets 1, 12, and 21, plus at the nut and saddle
  • String gauges and nut-to-saddle distance
  • Baseline string action, to plan for the overlay thickness

Pluck and velocity sensing: Roland GK-3

  • We selected the Roland GK-3 divided (hexaphonic) pickup, which gives a separate signal for each string
  • It mounts without drilling, so the stock bridge stays in place and each string’s ground path is preserved
  • It also makes the setup reversible, which suits prototyping
  • Our own ADC and firmware will read the six signals for pluck timing and velocity
  • Still to confirm: breakout box and cable, mounting position on the Tele, and ADC front end

Next steps

  • Finish the measurements and generate the pad layout
  • Order a test flex PCB, and prototype the scanning firmware on a bench board first
Posted on

Aaqib Status Report (10/3)

  • Research more types of piezo devices for the guitar sensors.
  • Rationale: Pickups are currently more than a $100. Even though we have the budget for it, I want to ensure that the saddles are the only option before I move ahead.
  • Circuitry schematic in the works.
  • Looking into methods of testing without matrix PCB for rationalizing if current design choices will be viable.
Posted on

Team Status Update 10/3

Trained and tested prediction model
Model Risks:
Next-chord prediction is way under the 80% top-1 target (best: 50.7%). Root-only accuracy is 58%, so the gap is probably mostly label noise, not model weakness.

Robustness to live timing jitter misses our target with a 29% relative accuracy drop vs. a 10% goal. We are considering ordering simultaneous notes deterministically before tokenizing, and training with more jitter variety.

Piezo Pickups:

Looking for a cheaper alternative to make an affordable product.
Piezo circuitry schematic close to completion.

Posted on

Jai Status Report 10/3

Some results from what I tested
Transformer overfit test: loss 0.014, confirming the training loop and model heads are correct.

Short training run (25.9M-param model, 6,000 steps, ~200M tokens): next-note top-1 accuracy 67.7% on clean input, 46.1% under timing jitter.

LightGBM next-chord baseline on the full 409,936-song corpus: 50.7% top-1 on next-bar chord, 46.8% on next-chord-change

 

Progress is ahead. Next week I hope to see if 80% goal is actually reasonable given the current scores. If i can get it up to ~65 i’ll keep trying for 80%

Posted on

Tim’s Status Report (9/26)

Found suitable donor guitar on Facebook Marketplace. Negotiating and planning to purchase this week.

Finalized overall design for the guitar’s fret matrix. Planning to implement in KiCad and order flex PCB after the precise measurements are taken of our guitar neck.

 

Posted on

Aaqib’s Status Report (9/26)

  • Researched piezo pickups and implementation to analyze drum’s frequency.
  • Considering different methodologies of pickup filtering for quality audio (and also so that I don’t create a crater on the Teensy).
  • Slightly behind schedule; however, I should be able to achieve required implementations which include piezo mounting, wiring and testing along with calibrations.
Posted on

Team Status Update 9/26

Model Risks:
Low Accuracy or high accuracy with too low latency. This would be mitigated by trying to predict longer into the future. but also risks user changing in the middle of the prediction. However since usecase is for beginners, this should more than likely not be a problem since chords will be predictable.

Fret sensing + Piezo electric sensors:

Complete plan has been finalized. Moved on designing the PCB for the guitar neck and firmware design. Quotation and part finalization will happen this week. Not on schedule, however plan was reworked and we are on schedule according to new plan. The new plan also takes into account better parallelization in work.

Everything else is still on schedule.

Tim: Part A: SoloMuse can reduce the isolation of solo practice by giving a user the felt experience of playing with other musicians, which has a real psychological benefit for motivation and enjoyment. SoloMuse uses embedded electronics inside a physical instrument a user holds against their body. This will be a low-voltage design with no exposed high-current paths. The fret matrix will be sealed/insulated so a broken string or sweat doesn’t create a short. SoloMuse lowers the economic and social barrier to a creative/expressive outlet, which matters most for people who otherwise have no path to it (no access to instruments/lessons/bandmates)

Aaqib: Part B: Considering social factors, it is significant to ensure that the piezo pickups along with the teensy can adapt to different cultural music playing. That is, different playing patterns induce different electrical oscillations that can be really aggressive therefore inducing voltages high enough to destroy the teensy if not carefully filtered. An example with aggressive playing includes the flamenco.

Jai: Part C: In regards to economic factors the product is meant to run locally on a user’s laptop. Thus the barrier to entry for model generation is almost non existent given users have a relatively newer laptop. The guitar is built using a donor guitar. In a high volume production environment, this cost would be more than a regular guitar, but no so much more that it would be out of reach to a normal user. A normal user in theory would be able to go to the store, buy the guitar, download the application and plug it in, letting it work without much hassle. Because of the hardware needed to add to the guitar, it is not feasible to have them install it themselves, but given guitar players often buy multiple guitars for multiple purposes this would not be a problem.

Posted on

Jai 9/26 Status Report

Began training on custom dataset for next note prediction. Using LightGBM as  model. Trained on all instruments in order to teach the model music theory. Will finetune on guitar specifically after base training reaches acceptable level.

Did research and choose the options I will run tests on for different model options. Features are still up to testing and I will change those more than the model options

Small test result

Progress is on schedule. Plan to finish the larger training and analyze results. Will experiment with many different optimizations and features in order to increase accuracy. Should run many different ablations next week to test accuracy and inference latency.

Ran on the AMD GPUs in the 1305 clusters. Some stats

GPU benchmark matmul throughput bf16 122 TFLOP/s, fp32 1.2 TFLOP/s (~100x gap)
Test suite 16 unit tests all pass on-box
Overfit test 1,000 fixed windows, 3,000 steps loss 18.1 → 0.012; guitar/bass/drum head accuracy 99.8%+
Training slice 18,284 songs, 6,000 steps, 197M tokens (~1 epoch of a 5% slice) held-out loss 1.79, next-note top-1 67.5% (clean), 47.6% (±30ms jitter)

Memory / throughput

Peak VRAM: 7.6 GB of 32 GB (24%)
Step time: 224 ms for batch 32 × 1024 tokens
Training throughput: ~146,000 tokens/s
Model size: 25.9M params (6-layer decoder, RoPE + SwiGLU)
bf16 check: every matmul in forward+backward confirmed bf16, zero fp32 fallback