Posted on

Team Update 4/10

This week, we mainly focused on the interim demo. In preparing for the demo, we noticed that our Gantt chart needed to be updated. We have attached the new version here. This creates more specificity on the tasks we are working on and includes some of the delays we have had in reaching our goal.

One setback we had the morning of the demo was the integration between the game code itself and the drum module. Because the first time that this error happened was on the morning of the demonstration, we are still working on debugging it and ensuring that this does not happen again. Furthermore,  because the integration between Windows and Mac for the sound library we want to use for the game seems very difficult to transfer between platforms, we are working on picking and implementing a new package for this. Finally, on the creation of the beat map side, the actual beats are able to be created in a way that is somewhat similar to the song. However, more testing needs to occur. This is further explained in the individual post.

A large portion of our work is on aesthetic changes. The game play code (while mostly playable) had to be updated to reflect a more interesting user experience. The specific changes can be found in the individual team diagrams. Furthermore, the drum apparatus will now have lights to also create a more interesting user experience, so this is also being worked on.
Schedule wise, we are on track to follow the new gantt chart. We do not foresee any reason why we won’t be able to follow this schedule, so we should be able to have a playable game at the end of the semester!

Posted on

Shreya’s Update 4/10

This week, I was mostly focused on the Interim Demo. Last week, I had the idea of using the STFT to separate the audio file into frequency bins to analyze through the intensity analysis. I thought this would be more efficient due to the variability of the number of bins. However, I was having some trouble implementing it.

I spoke to Professor Sullivan regarding the approach, and he mentioned that it would be better to use bandpass filters and then run an intensity analysis on that. I ended up implementing that with scipy and then running the intensity analysis on the filtered wav files.
On Wednesday, I had an odd bug where there would be no outputs during long stretches of time during the audio file. I was able to fix that by scaling the array, and then analyzing it. That issue is fixed.

Timing wise, while the algorithm takes about six minutes to analyze a three minute audio file, I should be able to get this down to three minutes as per the user requirements through streamlining the code. I will not be doing this for this week because this will likely be a finishing touch after tuning and testing the algorithm.

Currently, the beatmap seems to have an offset correlating to a song. George suggested adding a noise to the audio file whenever my algorithm detects a beat, so I should have that done by Wednesday so I can better understand whether there is an offset- or what exactly is happening.
I am fine on schedule. We have had to change the Gantt Chart which reflects our scheduling differences. The updated Gantt will be on this week’s team update.

 

Posted on

George’s Status Report for 4/10

Here is a list of tasks that I completed this week:

  1. Designed main menu
  2. Created and added art to gameplay screen
  3. Created sounds for main menu and song selection menu
  4. Created song selection menu
  5. Added fading screen transition between song selection menu and gameplay
  6. Added hit indicators that show how the accuracy of a player’s hit
  7. Fixed small bugs with sound loading and image loading

I worked mainly on SongSelectionMode.cpp, Sound.cpp, PlayMode.cpp, and FadingScreenTransition.cpp this week.

Here is a video of the current state of the game:

As you can see from the video, there is an issue with the font rendering. The text is not being shaped correctly by Harfbuzz. It looks like the lowercase letters are placed at weird locations, which makes the text look a bit silly. I’ve known about this issue for a few weeks now but I have not had the time to go back and fix it. I’m hoping that I will have the time to fix it once we start playtesting, which will happen the week of April 19.

Here is a list of sounds that I’ve created to make the gameplay feel more polished:

Here is the main menu design. I used GIMP (a free image editing software) to create the background and arrange the menu together. I used Google Drawings to create the title text and button text.

This week I did not get to all of the tasks that I had originally planned for this week. I wanted to continue working on Windows compatibility, but after discussion with the Professor and our TA during the Interim Demo we decided to not pursue Windows compatibility because it was taking too long. For now, our game will only run on MacOS.

By the end of this week I would like to have our game in a completely playable state. We need to begin playtesting soon, so it is critical that that game is fully playable by the end of next week. This will give us 1 full week to do playtesting and add any final polishing touches. I am still on track to complete all of my goals before the end of the semester.

Posted on

George’s Status Report for 4/2

This has been a very hectic week for me. The good news is that I have job offers on the table for after I graduate! The bad news is that I had a little less time to work on this project than I would have liked. I had job interviews on Wednesday and I’ve had a ton of phone calls and decision-making to do this week, which was pretty time consuming.

The game is now compiling on Windows. I met with my teammates last week and we all worked together to resolve the compilation problem. To get the game to compile on Windows, we had to comment out the Sound.cpp code, because the sound library that I am using, which claimed to work on Windows, does not work out-of-the-box on Windows.

The game is in a-playable state, but it is still pretty simple. The remainder of my work for this semester will be related to making the game more polished and fun to play.

Here are the tasks that I will be working on next week. This is a lot of work; I intend on working a little bit during Carnival. It is critical that the game takes shape ASAP so that we can playtest it after carnival.

  1. Get sound to work on Windows
  2. Add “Miss”, “Good”, and “Awesome” hit indicators to PlayMode
  3. Add health bar to gameplay scene
  4. Set up basic main menu and song selection menu so that Shreya and Stephen can work on them
Posted on

Team Status Report for 4/2

Over the course of the previous week, the two primarily tasks we’ve worked on as a team were the modification of the C++ codebase so that it could function (and therefore be useful in testing) on Windows machines and the integration of our individual project components into a functional device that replicates the end-goals of our project.

Regarding the C++ codebase transfer, we wanted to make sure the codebase was operational on Windows machines, as up until this week we had only ever tested it using a Mac, and being able to run the code on Windows would make certain integration testing steps (such as Shreya testing Json files in the program) much easier. Unfortunately we did run into some issues derived from specific libraries, and although we have yet to resolve those issues for every library currently in the Mac version, the only ones we have yet to fully integrate aren’t integral to the code’s operation, so it was possible for us to display the beatmaps as a series of notes in the C++ game. The problematic library that we had to remove was related to audio, which is an integral part of our game’s design even if the code can run without it, so in the following weeks we hope to resolve that issue as well.

Regarding the integration of our individual modules, we’ve connected the three individual subsections (Shreya’s beatmaps, George’s C++ code, and Stephen’s drum module) into a singular system so that we can verify the full information stream of our project. This is in preparation for the interim demo next week, during which we hope to show an unpolished version of how we intend for our final product to function. Going into next week, we hope to further polish our system with improved beatmaps, visuals, and hardware modules before the interim demo, but we also need to make sure we don’t include any new/developing features into our system that we haven’t had the time to verify the accurate functioning of yet, as they could fail unexpected during the demo, preventing us from demonstrating the parts of our project we have functioning/meeting our expectations.

Regarding how this progress fits into our overall timeline, we are on pace with our expectations, but there still exists a good amount of polishing we need to perform before we’d feel comfortable calling our project a “game” like our end of semester target requires.

Posted on

Shreya’s Update 4/2/2022

This week, I was working on getting the gameplay code installed on my computer. That worked around Wednesday, so I was able to begin visualizing the beat maps.
For the second half of the week, I started looking at melody tracking algorithms. I figured out how to map the frequency bands to the timing with a STFT (graph shown below).

I’m working on extracting the melody itself and getting the time stamps. According to this paper, the algorithm is to determine the frequency of the melody line (orange line on the bottom of the graph) and then extract the time stamps from those frequencies. I’m currently looking at using an external package to get these frequencies, but I should have a weak melody tracker by Monday.

For scheduling, I should be fine. I’m working on the automatic part of the beat tracker and the melody tracker simultaneously, so I’m pretty much on schedule.

Posted on

Stephen’s Status Report for 4/2

Over the course of the last week I’ve been primarily working on getting the C++ code to work on my Windows computer, as well as prepping for the upcoming interim demo on Wednesday 4/6.

Although last week I mentioned that I was expecting the PCBs I had ordered to arrive, I had actually misread the manufacturing completion date as the shipping arrival date. The true shipping arrival date turned out to be 4/4, so I haven’t gotten my hands on the PCBs yet, which has prevented me from soldering the components and printing the final drum design like originally planned.

As such, I’ve mainly been working on other minor tasks in preparation for next week. For example, both I and Shreya spent some time working with Visual Studio 2022 in an effort to get the C++ code working on our Windows machines. We did manage to get it working eventually, but only after removing an audio library, which will be necessary in our final product and so we’ll need to find a way to incorporate it back into the Windows version of the codebase later.

Additionally, I’ve been prepping the demo drum module for use in the interim demo. A video of such can be seen here:

When pressing the drum, the ESP32 simultaneously sends a serial signal to the computer (to be read by the C++ code) while also powering the feedback LEDs that give users a physical means of knowing their hit was detected. Other than improvements to the design/layout, this is exactly how the final drum module is planned to function.

Other than those two tasks, I’ve also been working on some modifications to the ESP32 code to handle multiple drum inputs as well as designing some drafts of what the ESP32 housing will look like. Due to the cost and time required to get custom PCBs, that likely won’t be feasible for the ESP32 module. As such, I’m planning on using a combination of breadboards and internal wiring to achieve the needed stability and connection quality with the ESP32.

Regarding where I feel this progress is at, I think I’m on schedule as what I have currently is sufficient for presentation during the interim demo next week. Although I would’ve liked to have the final drum module/PCB printed, soldered, and assembled together by then, I may still be able to do so depending on when the PCBs arrive next week. Once I do get that final module printed and assembled, most of the remaining objectives I’ll have going forward will be related to integration and helping out on other tasks where I can.