Course: Carnegie Mellon ECE Capstone, Spring 2026 | Members: Abishek Anand, Caleb Song, Rishabh Cowlagi
This week, I worked on upgrading some of the UI elements. For example, adding a more interactive tutorial page that makes the user exp more enjoyable for the public demos. I also worked on fixing a graphics rendering bug that would sometimes cause the game to need to be restarted. Overall, I would say we are on track for the public demo.
Unit tests: end to end network latency, end to end drum pad to daisy seed voltage reading latency, double hit rate
System testing: overall user testing
This week, I mostly worked on preparing for the final presentation. Part of this included flushing out the unit tests which I reported on in class.
The biggest new tool to get used was definitely the Bevy game engine. I had to spend a lot of time on the Bevy official docs page to get up to speed and be able to make changes to the code. My main learning strategy was to “learn by doing”. I would jump into the code and consult the docs as I went when I was confused.
This week I’ve been working on making significant improvements to the game engine to make the experience a lot better for the users. This will help us during the final demo to keep the audience engaged. The changes involved adding a tutorial to explain the basic mechanics of the game, adding predetermined levels 1 and 2 for easy and medium difficulty, and adding a delete button to delete user added songs.
I also want to explore allowing a user to upload mp3 files as well as .wav files since mp3 is more common (right now we only support .wav)
Screenshots of the main UI changes:
Tutorial:
The main next steps are to get multiplayer working via networking. I would also like to do more user testing to figure out what more improvements can be made to the UI and game experience to make the demo more interesting.
This week, I contributed across different parts of the project as we continued progressing toward a more complete and integrated system. I was involved in supporting both the hardware and software efforts, particularly during testing and integration of the current 3-pad setup.
On the hardware side, I helped with general testing and debugging of the pads, including observing issues like double hits and signal inconsistencies and working with the team to think through possible fixes such as thresholding and dampening. On the software side, I assisted with validating the pipeline end-to-end, making sure that inputs from the pads were being reflected correctly in the game and identifying areas where behavior was not fully consistent.
I also helped with broader system-level testing, including trying out the current UI and gameplay experience and giving feedback on areas that could be improved, as well as thinking through how the system will scale to support additional pads and multiplayer functionality.
Overall, I’ve been helping where needed to keep progress steady across both hardware and software. Moving forward, I plan to continue supporting testing and integration as we expand to more pads and begin working more seriously on networking and multiplayer.
On the software side, I contributed to improving overall system robustness and usability. This included helping refine parts of the game pipeline, supporting UI improvements, and assisting in evaluating and integrating different approaches for drum transcription to improve performance on more complex songs.
Overall, I’ve been helping wherever needed to keep both sides of the system moving forward together. By next week, my goal is to continue supporting integration efforts, help refine system stability as we move toward 6 pads, and assist with early planning and implementation of the networking/multiplayer components.
This week I helped on both the software and hardware side of the system. On the software side, I helped with integrating the hardware interface to process inputs from the drum pads and integrate with the game logic. I also helped with getting the Omnizart model working for drum transcription.
On the hardware side, I helped debug the pad system to get more reliable readings.
We have a rough end to end game working, and now the goal is to have it be more refined. This involves scaling up the number of pads we have, making the pad casing more aesthetic, syncing the music properly in the game, and improving the game UI so that players have a more enjoyable experience.
This week I worked on both the hardware and software components of the project. On the hardware side, I spent time in the lab assembling and testing the drum pad system, helping ensure the pads were correctly wired and able to send hit signals through the transmitter to the central hub. I also helped with initial calibration and testing to verify that hits were being detected reliably without excessive noise. On the software side, I continued developing the game and learning platform, including improving the user interface and working on the note identification pipeline. This included integrating new components into the software stack and beginning to connect the parsing system with the gameplay logic. Overall, I spent a significant amount of time this week working across both sides of the system to move us closer to full hardware–software integration.
This week our team made progress on both the software and hardware components of the project. On the software side, significant time was spent debugging the build infrastructure to successfully integrate Omnizart, the machine learning model we plan to use for note identification, into our existing system. After resolving several complex build issues, Omnizart was successfully compiled, and work has begun on transitioning our current note identification pipeline to use this model, though the full pipeline is not operational yet. We also improved the user experience for the learning platform by creating a home page and a page that allows users to upload songs to be parsed, which also begins to support future hardware integration. On the hardware side, the drum pad system has been successfully linked to the transmitter and is now able to send hit intensity levels to the central hub. Sensitivity calibration is currently being performed, and the voltage threshold will likely be increased from 50 mV to around 80 mV to prevent noise from being interpreted as hits. Work has also begun on implementing the Multiceiver functionality of the Nordic nRF24L01+ to allow the receiver to handle messages from multiple pads simultaneously, as well as initial development of the DSP pipeline so audio feedback can be heard by the user. While some hardware work is slightly behind schedule, additional lab time this week should allow us to complete calibration, measure end-to-end latency, and begin integrating the hardware and software systems together.
This week I was mainly working on writing the design report. After meeting with HK, we also decided to move away from the ML solution from drum transcription in order to keep things simpler.
In writing the design report, I narrowed down the overall structure of the game, the pass thresholds, variables that will be tracked, songs that will be played, etc.
I’ve also gotten to work on building the game engine in Rust. Currently I have a basic Piano Tiles like set up which can move “tiles” down the screen every few seconds. The next steps would be timing this with the actual drum notes in our preloaded songs and also figuring out MIDI communication so that the game can interface with the hardware side Caleb has been working on.
Rishabh and I ran into a fairly interesting DSP problem while working on the drum extraction logic in the software architecture. Our original instinct was that drums are just certain frequencies, right? That’s why Rishabh was able to implement a deterministic FFT to extract the 3 main drum sounds.However, we realized we actually wanted the full spectrum of drum sounds (up to 13) and this introduces a lot more complexity. Real drum kits are actually quite messy since multiple sounds can be occurring simultaneously. This causes the waveform to be a sum of sources, and there is no way for us to know what the original sources were with just a simple filter. The entire problem of drum transcription becomes mathematically ill-posed. We need some way of getting strong priors to actually predict what the original sources should be. This is where machine learning comes in.
Omnizart is a python library that solves exactly this problem. This project includes a pretrained model that is meant to optimally transcribe drum sounds from an input .wav file. I was able to write some inferencing code that loads the models weights, takes in a .wav file, and outputs the models predictions for the drum sound at every timestep. However, I am still running into some issues running this code due to some environment conflicts.
Given this progress, I feel like I am on schedule. The next steps for me would be to integrate this into the rest of the codebase and finding out a way to streamline inferencing the model with running the rest of the game engine.