Posted on

Shreya’s Update 4/16

Unfortunately, this week, I was pretty sick so I was absent from class.

However, I was able to make significant progress with the accuracy of the beat map creator. I ended up changing the energy calculation by making the window smaller than what it was before. I was calculating instantaneous energy in bins of 43 (recommended by the source I was using), but I ended up making the bin of length one instead. This also ended up being significantly faster. Currently, I am analyzing two frequency bands and splitting that into four buttons. I am analyzing the frequency bands separately and for a three and a half minute audio file, it takes about 112 seconds per frequency band to analyze. While that is slightly above my user requirement, tomorrow I plan to condense the algorithm into one loop so hopefully, it should take about 200 seconds total for a single audio file.

Also, as I mentioned last week, (George’s idea), I added a sound to indicate when my tracker found a beat. I have attached a few sound samples before. I have been creating a few additional beat maps as well for the testing phase of our game that we will be going through in a few weeks.

For the progress for the next week, I need to create a lot more files for testing as well as for verifying that my user requirements are met (or where they fall short). Furthermore, I need to create a main function to call everything instead of having to do it myself. This should only take a few hours because the code is pretty modular. Hopefully, I can have all of this done by Wednesday.

For scheduling, I should be on track. Everything is pretty much done, but I am working on some final touches.

NOTE: I’m having trouble uploading the files to the website because it says the maximum size of the files has been exceeded. I can show them later.

Posted on

Stephen’s Status Report for 4/16

The majority of this week was spent creating the 4 final drum modules, which included soldering each of the PCBs and 3D printing the drum housings. As of right now, all of the PCBs are soldered and (mostly) functional and I’ve printed 3 of the 4 total drums (the dremel printers in TechSpark were occupied most of this week so I’m printing the last one this weekend).

https://drive.google.com/file/d/1T9HWiaXTrl7rABxwUqzvM6yQnoAtga9A/view?usp=sharing

https://drive.google.com/file/d/1MaeZnkecdY8ms-ItmZ9qRBaB1ymTffSX/view?usp=sharing

Currently, 2 of the 4 PCBs I’ve soldered are operating perfectly, with all 4 LEDs receiving feedback from the ESP32.

https://drive.google.com/file/d/1JVWyY8zhrED-C6ImZAhHwnd7PPoJRTvZ/view?usp=sharing

https://drive.google.com/file/d/1JCIqvQrxr8q2foaXEXLrCToc0U5jX7bv/view?usp=sharing

 

However, the two other PCBs have minor soldering issues with some of their LEDs, so I haven’t trimmed their leads and placed them into drums yet, as I want to resolder their connections properly.

https://drive.google.com/file/d/1CVYEODAB1FGnWKjktggWexWcM0r705bB/view?usp=sharing

https://drive.google.com/file/d/1Js0cr5h1dcjLEnHlIN9yyXimBuaWOt4X/view?usp=sharing

The issue which caused this is regarding the small solder beads depicted below, as all 4 LEDs on the PCBs worked immediately after soldering, but upon bringing them back to my dorm I realized that the LED leads were jostled in my backpack, which caused the hardened solder to break itself off from the PCB, ultimately disconnecting it from the circuit. To fix this, all I need to do is reheat the solder and form a better (more shock resistant) connection.

https://drive.google.com/file/d/1UG_pr4wPE5gjd6lG7UZG1uREZa2nyTdW/view?usp=sharing

Lastly, I also acquired some wooden bases and supports for the drum modules this week. These bases will be fixed to the bottom of the PCB and to the drum’s base, which will provide support and isolate users from the drum internals. Additionally, I opted to use wood for this purpose as it was cheaper than 3D printing these parts and by using wood the bases will have more friction with whatever surface they are placed on, which should help prevent them from sliding around as much.

https://drive.google.com/file/d/1WeOysxl3Cy4t8Gs_CMU4My-Iigs1XabT/view?usp=sharing

https://drive.google.com/file/d/16U3gzAclyG5kQM6xwUeATo90310gCExV/view?usp=sharing

Regarding where I am in regards to our schedule, I think I’ve caught back up on pace this week/weekend, as once I’ve finished printing the last drum and resoldering those PCBs, the drums themselves will be in their finished state (barring some aesthetic improvements possibly) which will allow me to perform some needed testing this week. I do also need make a housing for the ESP32, but that will be purely to isolate it from the user rather than having a major functional purpose, so all I really need to do there is fix the ESP32 in a small box with holes for it’s wires, which shouldn’t take much time.

Posted on

George’s Status Report for 4/16

49I did a lot of work on this capstone project last week (Week of April 3), and I had deadlines in 2 other classes this week, so this week I worked on this capstone project less than I typically do. That being said, I did make some progress this week:

  1. Fixed bug related to drums not being recognized sometimes
  2. Made preparations for playtesting next week

Earlier this week we were running into a serial message parsing issue with our gameplay code. I spent some time debugging this and I came to the conclusion that there was a mismatch between Stephen’s microcontroller serial code and my gameplay serial code. The issue was caused by an erroneous “\n” character being sent from the microcontroller. Once we removed the “\n” the issue fixed itself.

I also created a Google Form where people can sign up to playtest our game. We will be having multiple playtesting sessions next week. I will be present at all of these sessions because the game is being run on my computer.

I am on progress to meet the end of semester deadline. Our game is nearly complete; the remaining features are mainly cosmetic.

Next week I will conduct playtesting sessions, add some finishing touches to the gameplay GUI, and prepare for the final presentation.

Posted on

Stephen’s Status Report for 4/10

The majority of this week was spent preparing to print the final drum modules, as earlier this week the PCBs arrived from PCBway, and after soldering the components to verify that they perform as expected, I was ready to 3D print the final drum design. The only change I made from my recently posted drafts was to reduce the thickness of the top wall of the housing from 5 mm to 2 mm. This is mainly to allow the LEDs more room to stick through the holes in this layer, as I hadn’t originally accounted for how the stubs of the other through-holes components may prevent them from getting as close to the roof of the housing as the PCB would naturally allow.

The print had reduced scaffolding and height compared to the demo design, which brought the cost down from $56.50 previously to just $36.00 for this final version.

However, after clearing the scaffolding, I encountered an issue with the design’s size, as although both the drum’s internal cavity and the PCB measured 11 cm, the PCB was slightly too large to fit within the drum.

Since this size difference was less than a millimeter, I’ve been trying to sand down the interior of the drum enough to allow the PCB to fit. However, although the difference in size has shrunk from my efforts, the PCB is still unable to fit smoothly even after several hours of sanding.

I still believe this drum module is salvageable, as I was able to shorten the difference greatly, however I’ll need to dedicate more time to sanding or use a tool within TechSpark to assist me in doing so. Going forward, I plan on reducing the side walls of the housing from 5 mm to 3 mm for the other three drums, with all of the gained space going to the inside of the module. This may give me too much room on the drum’s inside, however it will be much easier to remove the additional space rather than to create more through sanding.

Regarding where I am in regards to our schedule, I am now slightly behind where I had hoped to be after carnival, as I was hoping to have a fully functional final module ready going into this week. However, This is not a major delay, as if I am able to perform a successful 3D print tomorrow with the new design I’ll only be a day behind where I expected to be. Other than that, the rest of this week will be focused on finalizing the drums and the Arduino code which communicates with them, so that hopefully I’ll be able to assist with other tasks going into the last few weeks.

Posted on

Team Update 4/10

This week, we mainly focused on the interim demo. In preparing for the demo, we noticed that our Gantt chart needed to be updated. We have attached the new version here. This creates more specificity on the tasks we are working on and includes some of the delays we have had in reaching our goal.

One setback we had the morning of the demo was the integration between the game code itself and the drum module. Because the first time that this error happened was on the morning of the demonstration, we are still working on debugging it and ensuring that this does not happen again. Furthermore,  because the integration between Windows and Mac for the sound library we want to use for the game seems very difficult to transfer between platforms, we are working on picking and implementing a new package for this. Finally, on the creation of the beat map side, the actual beats are able to be created in a way that is somewhat similar to the song. However, more testing needs to occur. This is further explained in the individual post.

A large portion of our work is on aesthetic changes. The game play code (while mostly playable) had to be updated to reflect a more interesting user experience. The specific changes can be found in the individual team diagrams. Furthermore, the drum apparatus will now have lights to also create a more interesting user experience, so this is also being worked on.
Schedule wise, we are on track to follow the new gantt chart. We do not foresee any reason why we won’t be able to follow this schedule, so we should be able to have a playable game at the end of the semester!

Posted on

Shreya’s Update 4/10

This week, I was mostly focused on the Interim Demo. Last week, I had the idea of using the STFT to separate the audio file into frequency bins to analyze through the intensity analysis. I thought this would be more efficient due to the variability of the number of bins. However, I was having some trouble implementing it.

I spoke to Professor Sullivan regarding the approach, and he mentioned that it would be better to use bandpass filters and then run an intensity analysis on that. I ended up implementing that with scipy and then running the intensity analysis on the filtered wav files.
On Wednesday, I had an odd bug where there would be no outputs during long stretches of time during the audio file. I was able to fix that by scaling the array, and then analyzing it. That issue is fixed.

Timing wise, while the algorithm takes about six minutes to analyze a three minute audio file, I should be able to get this down to three minutes as per the user requirements through streamlining the code. I will not be doing this for this week because this will likely be a finishing touch after tuning and testing the algorithm.

Currently, the beatmap seems to have an offset correlating to a song. George suggested adding a noise to the audio file whenever my algorithm detects a beat, so I should have that done by Wednesday so I can better understand whether there is an offset- or what exactly is happening.
I am fine on schedule. We have had to change the Gantt Chart which reflects our scheduling differences. The updated Gantt will be on this week’s team update.

 

Posted on

George’s Status Report for 4/10

Here is a list of tasks that I completed this week:

  1. Designed main menu
  2. Created and added art to gameplay screen
  3. Created sounds for main menu and song selection menu
  4. Created song selection menu
  5. Added fading screen transition between song selection menu and gameplay
  6. Added hit indicators that show how the accuracy of a player’s hit
  7. Fixed small bugs with sound loading and image loading

I worked mainly on SongSelectionMode.cpp, Sound.cpp, PlayMode.cpp, and FadingScreenTransition.cpp this week.

Here is a video of the current state of the game:

As you can see from the video, there is an issue with the font rendering. The text is not being shaped correctly by Harfbuzz. It looks like the lowercase letters are placed at weird locations, which makes the text look a bit silly. I’ve known about this issue for a few weeks now but I have not had the time to go back and fix it. I’m hoping that I will have the time to fix it once we start playtesting, which will happen the week of April 19.

Here is a list of sounds that I’ve created to make the gameplay feel more polished:

Here is the main menu design. I used GIMP (a free image editing software) to create the background and arrange the menu together. I used Google Drawings to create the title text and button text.

This week I did not get to all of the tasks that I had originally planned for this week. I wanted to continue working on Windows compatibility, but after discussion with the Professor and our TA during the Interim Demo we decided to not pursue Windows compatibility because it was taking too long. For now, our game will only run on MacOS.

By the end of this week I would like to have our game in a completely playable state. We need to begin playtesting soon, so it is critical that that game is fully playable by the end of next week. This will give us 1 full week to do playtesting and add any final polishing touches. I am still on track to complete all of my goals before the end of the semester.

Posted on

George’s Status Report for 4/2

This has been a very hectic week for me. The good news is that I have job offers on the table for after I graduate! The bad news is that I had a little less time to work on this project than I would have liked. I had job interviews on Wednesday and I’ve had a ton of phone calls and decision-making to do this week, which was pretty time consuming.

The game is now compiling on Windows. I met with my teammates last week and we all worked together to resolve the compilation problem. To get the game to compile on Windows, we had to comment out the Sound.cpp code, because the sound library that I am using, which claimed to work on Windows, does not work out-of-the-box on Windows.

The game is in a-playable state, but it is still pretty simple. The remainder of my work for this semester will be related to making the game more polished and fun to play.

Here are the tasks that I will be working on next week. This is a lot of work; I intend on working a little bit during Carnival. It is critical that the game takes shape ASAP so that we can playtest it after carnival.

  1. Get sound to work on Windows
  2. Add “Miss”, “Good”, and “Awesome” hit indicators to PlayMode
  3. Add health bar to gameplay scene
  4. Set up basic main menu and song selection menu so that Shreya and Stephen can work on them
Posted on

Team Status Report for 4/2

Over the course of the previous week, the two primarily tasks we’ve worked on as a team were the modification of the C++ codebase so that it could function (and therefore be useful in testing) on Windows machines and the integration of our individual project components into a functional device that replicates the end-goals of our project.

Regarding the C++ codebase transfer, we wanted to make sure the codebase was operational on Windows machines, as up until this week we had only ever tested it using a Mac, and being able to run the code on Windows would make certain integration testing steps (such as Shreya testing Json files in the program) much easier. Unfortunately we did run into some issues derived from specific libraries, and although we have yet to resolve those issues for every library currently in the Mac version, the only ones we have yet to fully integrate aren’t integral to the code’s operation, so it was possible for us to display the beatmaps as a series of notes in the C++ game. The problematic library that we had to remove was related to audio, which is an integral part of our game’s design even if the code can run without it, so in the following weeks we hope to resolve that issue as well.

Regarding the integration of our individual modules, we’ve connected the three individual subsections (Shreya’s beatmaps, George’s C++ code, and Stephen’s drum module) into a singular system so that we can verify the full information stream of our project. This is in preparation for the interim demo next week, during which we hope to show an unpolished version of how we intend for our final product to function. Going into next week, we hope to further polish our system with improved beatmaps, visuals, and hardware modules before the interim demo, but we also need to make sure we don’t include any new/developing features into our system that we haven’t had the time to verify the accurate functioning of yet, as they could fail unexpected during the demo, preventing us from demonstrating the parts of our project we have functioning/meeting our expectations.

Regarding how this progress fits into our overall timeline, we are on pace with our expectations, but there still exists a good amount of polishing we need to perform before we’d feel comfortable calling our project a “game” like our end of semester target requires.

Posted on

Shreya’s Update 4/2/2022

This week, I was working on getting the gameplay code installed on my computer. That worked around Wednesday, so I was able to begin visualizing the beat maps.
For the second half of the week, I started looking at melody tracking algorithms. I figured out how to map the frequency bands to the timing with a STFT (graph shown below).

I’m working on extracting the melody itself and getting the time stamps. According to this paper, the algorithm is to determine the frequency of the melody line (orange line on the bottom of the graph) and then extract the time stamps from those frequencies. I’m currently looking at using an external package to get these frequencies, but I should have a weak melody tracker by Monday.

For scheduling, I should be fine. I’m working on the automatic part of the beat tracker and the melody tracker simultaneously, so I’m pretty much on schedule.