Posted on

Team Status Report for 4/4

We successfully were able to demonstrate our product with a working UI for our guitar hero game with 3 pads or 3 inputs. We were able to adjust the difficulty of the song on the spot, reduce the number of double hits and mis-inputs, and overall have a positive user experience.

Hardware: Our hardware was mostly working as expected without much noise detected however we did experience some scenarios where we did run into potential issues. (1) Our initial drum pad design had a case that wasn’t flush to our impact plate and thus residual hits would tamper with the impact plate and cause multiple hits to be registered. Furthermore this could be an issue with the overall drum not being tight enough. (2) Our newest pad did have some issues with double hits and as I described in Caleb’s Status Report, it could be an inherent characteristic of the acrylic impact plate. This can be accommodated with some software fixes and maybe some dampening circuitry. (3) Lastly, our drum frames are in direct contact with the table that it is on which gives rise to some noises being picked up from table hits and I believe it can be lightened with rubber stop gaps between frames but also between screw and the floor.

Hardware Test: Caleb will then conduct a series of 25 consecutive hits in 3 sets: control represents dead center hits, group 1 which consists of hits to the area between the center of the drum and the rim, and group 2 which consists of hits only to the rim. Then I will check the outputs on my serial monitor from the daisy seed and find the percentage of hits that result in a double hit being registered. From these fixes I am hoping to lower the percentage of ghost hits to <8% (only 2 ghost hits for every 25 hits). One thing to note is that there are some instances were double hits are intentional, if the grip on the drum stick is very loose causing the bouncing of the stick head on the drum pad.

Secondly, latency was a big concern of ours so I will hook up one end of an oscilloscope to the output of a piezo pin on the drum then I will hook up the other end to the tx port of the daisy seed to be able to determine the difference in time of arrival of each packets and ensuring that the time difference is within our limits of 10-20 ms with 5% tolerance. (9.5-21 ms)

Software: We got the overall game functional, the extraction of the drums from a song work well in regards to any type of music file inserted, however the pipeline does take a lot of time. Due to us having to convert a song into wav file, pushing it through demucs which takes 2 ish minutes to parse the drums out of. We manually went through one song and compared the drum music sheet with the notes being outputted to find there was a 95% match rate which is great news.

Some difficulties we faced was moving our ML model from an x86 proprietary driver model to a more open model which has helped streamline our process a lot. And also receiving ghost screens very seldomly.

Steps we can take for the future of our software interface is (1) improving the UI so that players can easily tell the difference and arrangement of pads, (2) improving the gameplay so that players know how well they are doing (numerical feedback system and normalized score tracker) so that they don’t lose interest, (3) implementing the networking interface so that we can simultaneously run 2 independent pad systems playing a single game on 2 different computers.

Software Testing: Most of our software testing will probably take the form of user experience feedback in the early stages because we want to improve the gaming aspect with a point system that is incentivizing for our users. Including checkpointing system to accommodate for beginner drummers. And then we will move on to stress testing our networked model with multiple data packets to ensure that our timing between client and server is synced and we do so without a lossy network.

Overall, we would say that we are on track to being able to finish our product by the final deadline but also on track with our schedule.

Posted on

Caleb’s Status Report for 4/4

The hardware aspect is mostly done and just needs to be reproduced to be able to make 6 pads. There are a few things that need to be modified however. We were originally using a plexiglass/acrylic material for our impact plate but making 6 of those were difficult due to the inability to laser cut them so we changed to acrylic which is a lot denser than the plexiglass. This does have its downsides because the dampening of the acrylic is most likely a lot less than the plexiglass. Therefore, there will most likely be a lot more double hits registered due to residual sound waves as I noticed in the interim demo.

A short fix for that is that I can increase the debouncing window on the arduino code to account for the residuals or increase the threshold for activation.

Some other variances with the hardware that I noticed was that the rims could register shots but there was low dampening from the environment which is not ideal. A fix for that is adding rubber pads on the screw heads of the pads so that the rubber can act as the dampener from the environment.

One concern is that I noticed that some of the piezos on the impact plates I just made had a very large negative voltage spike compared to other pads which were mostly net positive peaks. It could be an issue with the soldering but I will have to test it in the entirety of its configuration to be sure.

Lastly, for future steps I am planning to incorporate another daisy seed central hub with configurable settings such that 2 users are able to play multiplayer over a network with each daisy seed acting as a client server interface.

Posted on

Rishabh’s Status Report for 4/4

This week I completed most of the goals I had laid out from last week. Specifically, I improved much of the UI for the game by adding hit indicators and also visuals indicating how accurate any given hit was (similar to how Just Dance uses words like “Perfect!”, “Nice!”, or “Miss”).

Additionally, I started working on the summary page and completed the page for uploading songs (although there are still some graphical improvements that I would like to make to this page).

As for the networking part of the game, this is something that I’ve also spent the last week thinking through and planning. I do anticipate that it will require a relatively significant code rewrite process, so it’s something that I plan start making progress in this week.

Specifically, this coming week, I would like to finalize the single player UI with the game looking very professional and clean. Then, I would also like to make progress with the multiplayer networking part with the goal of finishing the preliminary version in the week after that.

Posted on

Abishek’s Status Report for 4/4

This week, I contributed across different parts of the project as we continued progressing toward a more complete and integrated system. I was involved in supporting both the hardware and software efforts, particularly during testing and integration of the current 3-pad setup.

On the hardware side, I helped with general testing and debugging of the pads, including observing issues like double hits and signal inconsistencies and working with the team to think through possible fixes such as thresholding and dampening. On the software side, I assisted with validating the pipeline end-to-end, making sure that inputs from the pads were being reflected correctly in the game and identifying areas where behavior was not fully consistent.

I also helped with broader system-level testing, including trying out the current UI and gameplay experience and giving feedback on areas that could be improved, as well as thinking through how the system will scale to support additional pads and multiplayer functionality.

Overall, I’ve been helping where needed to keep progress steady across both hardware and software. Moving forward, I plan to continue supporting testing and integration as we expand to more pads and begin working more seriously on networking and multiplayer.

Posted on

Abishek’s Status Report for 3/28

On the software side, I contributed to improving overall system robustness and usability. This included helping refine parts of the game pipeline, supporting UI improvements, and assisting in evaluating and integrating different approaches for drum transcription to improve performance on more complex songs.

Overall, I’ve been helping wherever needed to keep both sides of the system moving forward together. By next week, my goal is to continue supporting integration efforts, help refine system stability as we move toward 6 pads, and assist with early planning and implementation of the networking/multiplayer components.

Posted on

Team’s Status Report for 3/28

We successfully added 3 pads to the system and will try to accommodate 6 pads on one system because the nRF24L01+ system can support up to 6 active pipes on the multiceiver functionality. We are currently using channel 108 which is on the border of the free legal 2.4 GHz RF spectrum. This is to avoid any confusion and noise on the main 76 channel for 2.4 GHz which is going to be very saturated during demo day. Furthermore, with new materials coming in we can continue production up to 6 pads so we can split into 3 pads half and half for networked gaming hopefully. The hardware side is trying to implement DAC onto the central hub instead of the software side so that there could be a standalone aspect to the product but the practicality will be determined soon. The software system needs a few more tweaks to be able to support user input for songs and such.

We are pretty on track and that gives us room to try and tackle our stretch goals of creating a networked game for 2 different sets of drums. There will most likely have to be some encoding on the daisy to switch between the different contexts but otherwise there will be not much hiccup to upscale our product and we can find other ways to improve the sensitivity and accuracy of our product.

We just need to add our location calculations on the code to be able to determine the different types of sounds to play on the output.

Posted on

Caleb’s Status Report for 3/29

I have successfully developed most of the pad hardware to be easily recreated and will develop 6 pads within the next few weeks for our minimum viable product. Using the Arduino Pro Mini 3.3V allows for easy reprogramming to calibrate for any threshold modifications needed. Usually will have to do with time latency window in between hits as well as the minimum voltage threshold floor.  There usually might be very slight differences in tolerances because of the piezo disc manufacturing.

Finally, I will finish implementing digital playback audio on my central hub so that the system can be a stand alone product to be able to produce its own sounds. Furthermore, I will start implementing code on a separate daisy seed so that we can create a dual-networked system splitting the 6 pads into 3 pads for each player.

I would say I am pretty on schedule and in order for the whole team to be on schedule I will try to see in what areas I can help out with the assembly of the product on the software and networking side.

Posted on

Rishabh’s Status Report for 3/28

This week, both Caleb and I spent much of our time working on integrating both the hardware and software systems together. After some debugging and adjusting each of our interfaces, we got the integration working, so we’ve gotten to the point where we have an end to end basic product working.

Additionally, during the process of testing, I noticed that the ML model that we’re I was previously using (Omnizart) was doing a poor job at handing songs where the tempo was high, specifically with residuals from cymbals, so instead, I added a new drum transcription model (ADTOF) and then only register a cymbal note if both models register a note at that timestamp.

Finally, I also worked on improving the UI of our game, making a formal entry screen and a screen for uploading new songs (although this is still incomplete). Also, I added signals indicating when a specific note was actually played and an indicator showing if the timing of the hit note was off or not.

By next week, my goal is to get the UI in a much better place with it starting to look more ready for our final by completing the uploading songs screen and just overall making the UI look cleaner. Also, I would like to begin planning out the networking and multiplayer aspect of our game since I anticipate that this will require a significant redesign of some of the code.

Posted on

Abishek’s Status Report for 3/21

This week I helped on both the software and hardware side of the system. On the software side, I helped with integrating the hardware interface to process inputs from the drum pads and integrate with the game logic. I also helped with getting the Omnizart model working for drum transcription.

On the hardware side, I helped debug the pad system to get more reliable readings.

We have a rough end to end game working, and now the goal is to have it be more refined. This involves scaling up the number of pads we have, making the pad casing more aesthetic, syncing the music properly in the game, and improving the game UI so that players have a more enjoyable experience.

Posted on

Team’s Status Report for 3/21

We have been able to debug the end-to-end connection of our overall system, with 2 fully functional pads with the central hub being able to control the 2.4GHz communication between the pads and allow for near simultaneous perception of the inputs. We successfully debugged our “multiceiver” functionality of the nordic nRF24L01+ modules. These packets are parsed by the central hub and sent to the computer via UART and serial output which the software is able to process and use as inputs to our game to get a viable scoring gaming mechanic. We definitely have a lot more features that we want to include but overall, the latency falls within our target latency and it provides good gameplay. Furthermore, the translation of the songs and parsing into the gaming format is pipelined and provides good integrity on the overall file.

Some improvements that can be made is the synchronization of audio and gaming inputs could be further improved. We can definitely add a lot more pads to the configuration. We still need to implement more sophisticated learning paradigms to our gaming design, such as the checkpointing mentioned within our reports. Our end-to-end assembly took some time but overall there is little to no bugs due to comprehensive testing and planning prior to assembly. There will need to be more iterative designing of the pad casing but overall we are almost on track.

We will meet over the week to continue working in lockstep to be able to reach our goals of more functionality added to our overall design by syncing each other on our progress every 2 days and combining the systems.

Still running into a few bugs on the digital to analog conversion on the daisy seed and will need to revisit the documentation to be able to debug the current problems.