Posted on

Team’s Status Report for 4/25

Mostly this week we were focused on changing our debouncing logic on the hardware to avoid double hits and making sure the networking implementation is spotless and able to work under strenuous conditions. Finally, we also focused on making the design presentation and unit testing to make sure our system implementation meets our use case requirements.

HARDWARE UNIT TESTS:
Debouncing: In our default setting for 80 ms debouncing window and 60 mV activation threshold we found our erroneous double hit rate to reach around 27% for every 25 hits we played on the pad.

Therefore, Caleb ran a series of tests with 50 hits on each pad with 27% erroneous double hit rate as our baseline and control group to improve from.

Caleb worked with different activation thresholds from 100 mV to 200 mV because on preliminary testing on their pad we found that soft hits to register they reach around 100 mV and 60 mV is very similar boundary wise to hits from the environment. Changing this decreased the erroneous hit rate to about 23% showing it contributed to the problem but was not the root cause.

Then Caleb sweeped the debouncing time windows from 80 ms to 120 ms. Finding that overall erroneous inputs reach 6% and plateus around 110/115 ms thus he chose the 115 ms to be safe on the debouncing window.

Hit profiling:  Overall we achieved a 81% accuracy on the hit profiling but we got it to the 93% in our final presentation slides by switching our radius calculation equations.

Previously we used the below equation which was limited in the scope that we assumed that the sensors were placed perfect 120° apart and using sensor 0 as a reference point. While it did most of the general calculations correctly, we needed to get higher location pinpointing to make the system feel better for the user and thus we switched our equations to be a weighted function of the cosine of their angle offsets.

We swept each of the weights for the sensors using a series of tests with a ball  on a meter stick and locations marked within the center radius, middle radius, and edge radius (1.5 in, 3 in, 4.5 in) to measure the hit profiling and accuracy every 50 hits. We found for most of the pads, the weights to W0=0.92, W1 = 1.02, W2 = 0.90. (These results were arbitrary to the positioning of the sensors on the pads and their connections to the arduino but for the most part we needed to reduce the weights of 2 sensors because they were closer to each other than the other.)

 

Posted on

Caleb’s Status Report for 4/25

I have completed most of the hardware debugging and finishing the digital audio conversion on the hardware. Furthermore, the debouncing achieves a 94% accuracy on no erroneous double hit registers. So I am just planning to meet with the software side for one final run before integrating the system fully and recording the video. There are some difficulties in powering on each pad so I will try to augment the current design to make space for it later which was a shortcoming on my end as the designer. Otherwise we are looking pretty good in terms of integration and hardware capabilities.

I am mostly on schedule and am going to focus on adding the hardware details to the poster and the final design report. For future steps we could have been faster on the overall system wide integration to be able to pinpoint small bugs earlier; however, I feel like our unit testing on each of our own ends helped to make the transition as seamless as possible.

Posted on

Team’s Status Report for 4/18

After our interim demo we met up again to consolidate our next step goals because we were a little ahead of schedule and moved on to integrate the multiplayer aspect of our games. We have 2 daisy seeds which allows us to split up our hardware of 6 pads into 3 pads for each player. There were some set backs in setting up the client server conection and communication while staying true to the latency of the game. Furthermore, on the hardware side I found some optimizations by reducing the size of our hit data packets which allows us to increase the transmission speed without suffering too much packet loss from 128kb to 256kb packages reducing our perceived latency.

Overall, we are on schedule and have the remaining few weeks to debug any problems that arise with our final implementation and we have the last few weeks to touch up on our presentation and continue to test for any hidden bugs.

Some next steps would be more on the software implementation side to be able to make the interactive gameplay more beginner friendly by adding checkpoints and a normalized scorebar to create more incentive to keep combos of hits going.

 

Posted on

Caleb’s Status Report for 4/18

I just went over all the code within the hardware to make sure they had the most updated versions and did not register erroneous double hits. In the test/validation I described in the team status report I achieved a 97.3% no double hit rate on all pads by increasing the debouncing window between hits and slightly increasing the threshold of the hit activations.

Center hits are classified by having all piezo sensors reading within 1% margin of error from each other. Rimshot hits are classified as having 1 or 2 sensors being greater than 1 V and 1.5 V. An edge hit is classified as at most one sensor being greater than or equal to 1.5V

For center hit:  set at 100 bpm we achieved 100% no double hit rate,  set at 140 bpm we achieved 92% no double hit rate but 4% missed hits, set at 85 bpm we achieved 100% no double hit rate.

For rimshots: set at 100 bpm we achieved 92% no double hit rate, set at 140 bpm we achieved 88% double hit rate, set at 85 bpm we achieved 100% no double hit rate.

For edge shots: set at 100 bpm we achieved 88% no double hit rate, set at 140 bpm we achieved 83% no double hit rate, set at 85 bpm we achieved 96% no double hit rate.

We hit the target rates (<8% double hit rate) for center hits and rimshots but could not hit it on the edge shots which makes sense because I was not able to procure rubber stoppers in time for the final presentation so when you hit the edge of the drum casing, the vibrations are still preserved rather than being dampened and cause the piezos to pick up twice.

I finished up adding additional functionality to the central hub where it intakes keyboard inputs to set the mode to freeplay (audio generated by daisy seed on its own but needs a 5V 150 mA power supply), interactive game mode (interim demo but having the pads interface via bluetooth to the computer as inputs to a guitar hero-esque game), or multiplayer (a networked multiplayer game where 3 pads are split among the 2 players, each with their own central hub and laptop seeing the same game).

Overall, I would say I got a lot more comfortable looking at documentation especially on github repos and forums for open source resources. It also really consolidated my embedded understanding of nRF32L01+ protocols while trying to find a bandwidth of 2.4GHz WiFi that wasn’t too convoluted or noisy. Lastly, it also really helped with the overall design process; I feel like I practiced the idea of iterative designs well especially for the development of the pads which helped me to really streamline the process and get it done early so that I could focus on other features of the central hub. Also this helped consolidate my interpersonal skills when trying to meet deliverable deadlines as a team and gave me experience in being able to divvy up the work and trust my partners.

In terms of the schedule I am on schedule and have enough slack time to be able to debug anything that arises in final presentation day in time for the actual final presentation for capstone. For next steps, I will try to stress test the hardware to uncover any bugs that I may not be expecting.

Posted on

Team Status Report for 4/4

We successfully were able to demonstrate our product with a working UI for our guitar hero game with 3 pads or 3 inputs. We were able to adjust the difficulty of the song on the spot, reduce the number of double hits and mis-inputs, and overall have a positive user experience.

Hardware: Our hardware was mostly working as expected without much noise detected however we did experience some scenarios where we did run into potential issues. (1) Our initial drum pad design had a case that wasn’t flush to our impact plate and thus residual hits would tamper with the impact plate and cause multiple hits to be registered. Furthermore this could be an issue with the overall drum not being tight enough. (2) Our newest pad did have some issues with double hits and as I described in Caleb’s Status Report, it could be an inherent characteristic of the acrylic impact plate. This can be accommodated with some software fixes and maybe some dampening circuitry. (3) Lastly, our drum frames are in direct contact with the table that it is on which gives rise to some noises being picked up from table hits and I believe it can be lightened with rubber stop gaps between frames but also between screw and the floor.

Hardware Test: Caleb will then conduct a series of 25 consecutive hits in 3 sets: control represents dead center hits, group 1 which consists of hits to the area between the center of the drum and the rim, and group 2 which consists of hits only to the rim. Then I will check the outputs on my serial monitor from the daisy seed and find the percentage of hits that result in a double hit being registered. From these fixes I am hoping to lower the percentage of ghost hits to <8% (only 2 ghost hits for every 25 hits). One thing to note is that there are some instances were double hits are intentional, if the grip on the drum stick is very loose causing the bouncing of the stick head on the drum pad.

Secondly, latency was a big concern of ours so I will hook up one end of an oscilloscope to the output of a piezo pin on the drum then I will hook up the other end to the tx port of the daisy seed to be able to determine the difference in time of arrival of each packets and ensuring that the time difference is within our limits of 10-20 ms with 5% tolerance. (9.5-21 ms)

Software: We got the overall game functional, the extraction of the drums from a song work well in regards to any type of music file inserted, however the pipeline does take a lot of time. Due to us having to convert a song into wav file, pushing it through demucs which takes 2 ish minutes to parse the drums out of. We manually went through one song and compared the drum music sheet with the notes being outputted to find there was a 95% match rate which is great news.

Some difficulties we faced was moving our ML model from an x86 proprietary driver model to a more open model which has helped streamline our process a lot. And also receiving ghost screens very seldomly.

Steps we can take for the future of our software interface is (1) improving the UI so that players can easily tell the difference and arrangement of pads, (2) improving the gameplay so that players know how well they are doing (numerical feedback system and normalized score tracker) so that they don’t lose interest, (3) implementing the networking interface so that we can simultaneously run 2 independent pad systems playing a single game on 2 different computers.

Software Testing: Most of our software testing will probably take the form of user experience feedback in the early stages because we want to improve the gaming aspect with a point system that is incentivizing for our users. Including checkpointing system to accommodate for beginner drummers. And then we will move on to stress testing our networked model with multiple data packets to ensure that our timing between client and server is synced and we do so without a lossy network.

Overall, we would say that we are on track to being able to finish our product by the final deadline but also on track with our schedule.

Posted on

Caleb’s Status Report for 4/4

The hardware aspect is mostly done and just needs to be reproduced to be able to make 6 pads. There are a few things that need to be modified however. We were originally using a plexiglass/acrylic material for our impact plate but making 6 of those were difficult due to the inability to laser cut them so we changed to acrylic which is a lot denser than the plexiglass. This does have its downsides because the dampening of the acrylic is most likely a lot less than the plexiglass. Therefore, there will most likely be a lot more double hits registered due to residual sound waves as I noticed in the interim demo.

A short fix for that is that I can increase the debouncing window on the arduino code to account for the residuals or increase the threshold for activation.

Some other variances with the hardware that I noticed was that the rims could register shots but there was low dampening from the environment which is not ideal. A fix for that is adding rubber pads on the screw heads of the pads so that the rubber can act as the dampener from the environment.

One concern is that I noticed that some of the piezos on the impact plates I just made had a very large negative voltage spike compared to other pads which were mostly net positive peaks. It could be an issue with the soldering but I will have to test it in the entirety of its configuration to be sure.

Lastly, for future steps I am planning to incorporate another daisy seed central hub with configurable settings such that 2 users are able to play multiplayer over a network with each daisy seed acting as a client server interface.

Posted on

Team’s Status Report for 3/28

We successfully added 3 pads to the system and will try to accommodate 6 pads on one system because the nRF24L01+ system can support up to 6 active pipes on the multiceiver functionality. We are currently using channel 108 which is on the border of the free legal 2.4 GHz RF spectrum. This is to avoid any confusion and noise on the main 76 channel for 2.4 GHz which is going to be very saturated during demo day. Furthermore, with new materials coming in we can continue production up to 6 pads so we can split into 3 pads half and half for networked gaming hopefully. The hardware side is trying to implement DAC onto the central hub instead of the software side so that there could be a standalone aspect to the product but the practicality will be determined soon. The software system needs a few more tweaks to be able to support user input for songs and such.

We are pretty on track and that gives us room to try and tackle our stretch goals of creating a networked game for 2 different sets of drums. There will most likely have to be some encoding on the daisy to switch between the different contexts but otherwise there will be not much hiccup to upscale our product and we can find other ways to improve the sensitivity and accuracy of our product.

We just need to add our location calculations on the code to be able to determine the different types of sounds to play on the output.

Posted on

Caleb’s Status Report for 3/29

I have successfully developed most of the pad hardware to be easily recreated and will develop 6 pads within the next few weeks for our minimum viable product. Using the Arduino Pro Mini 3.3V allows for easy reprogramming to calibrate for any threshold modifications needed. Usually will have to do with time latency window in between hits as well as the minimum voltage threshold floor.  There usually might be very slight differences in tolerances because of the piezo disc manufacturing.

Finally, I will finish implementing digital playback audio on my central hub so that the system can be a stand alone product to be able to produce its own sounds. Furthermore, I will start implementing code on a separate daisy seed so that we can create a dual-networked system splitting the 6 pads into 3 pads for each player.

I would say I am pretty on schedule and in order for the whole team to be on schedule I will try to see in what areas I can help out with the assembly of the product on the software and networking side.

Posted on

Team’s Status Report for 3/21

We have been able to debug the end-to-end connection of our overall system, with 2 fully functional pads with the central hub being able to control the 2.4GHz communication between the pads and allow for near simultaneous perception of the inputs. We successfully debugged our “multiceiver” functionality of the nordic nRF24L01+ modules. These packets are parsed by the central hub and sent to the computer via UART and serial output which the software is able to process and use as inputs to our game to get a viable scoring gaming mechanic. We definitely have a lot more features that we want to include but overall, the latency falls within our target latency and it provides good gameplay. Furthermore, the translation of the songs and parsing into the gaming format is pipelined and provides good integrity on the overall file.

Some improvements that can be made is the synchronization of audio and gaming inputs could be further improved. We can definitely add a lot more pads to the configuration. We still need to implement more sophisticated learning paradigms to our gaming design, such as the checkpointing mentioned within our reports. Our end-to-end assembly took some time but overall there is little to no bugs due to comprehensive testing and planning prior to assembly. There will need to be more iterative designing of the pad casing but overall we are almost on track.

We will meet over the week to continue working in lockstep to be able to reach our goals of more functionality added to our overall design by syncing each other on our progress every 2 days and combining the systems.

Still running into a few bugs on the digital to analog conversion on the daisy seed and will need to revisit the documentation to be able to debug the current problems.

Posted on

Caleb’s Status Report for 3/21

This past week I was able to get the multiceiver functionality to work completely and able to register “simultaneous” hits to the drum pads. I also worked on developing the framing of the drums to make it look nice but also secure. I also finished the overall design including the foam dampeners added to the underside of the impact plate which greatly improved overall reading of the pad and isolation of signals from the pad. I also soldered extra wires onto these pads to improve signal integrity as well as the connection between pad and the breadboard.

We are on track for being able to show a somewhat viable demo by interim demo and I am on track to develop more pads for the final. Might have to tweak the 3D printing iterations to make a larger rim to secure the pad.

My next goals will be to create the housing as well as the digital sound output from the central hub processor itself. I was held up debugging the wireless multiceiver communication protocol.

This communication protocol can only support up to a max of 6 simultaneous mailboxes and then we can later tackle networking and being able to play across different location if time permits. But the immediate goal is to be able to create 6 pads and have it create a fault-proof end-to-end system by demo.