This week, I focused on testing our comparison algorithm and creating test videos and documentation to help prepare for the final demo and materials.
In terms of testing the comparison algorithms, I first started with creating a new video that was not too difficult to follow but not too simple to follow as well. I aimed to choreograph and film a simpler dance that an average person would be able to follow, still including some complexity so that we could test the effectiveness of the comparison algorithm. This allowed us to test each of the different comparison algorithms that we created and helped us understand where each of them was lacking and where each of them was succeeding. However, we ran into some issues with running just the comparison algorithm without the Unity integration, so our next step was to get a basic integration of the two systems to allow us to get quantitative and effective data on the effectiveness of our game, especially for our use case.
On Monday, we brainstormed ways on how to improve the user experience of the dance game, and one thing that we decided to add was a “ghost” overlay on the user avatar. We thought this would be helpful because the user can directly see what may be off from their dance moves compared to the reference video and also help them correct themselves if they are off from the reference video. After we were able to send each of the 5 comparison algorithm data to the Unity client, we needed to create videos of each of the algorithms in action to see which would be best suited for the final demo.
We used the reference video that I created earlier, and then I used our system to learn the dance and see how our comparison algorithms perform differently. After we were finished, we recorded videos of my dancing with the system, analyzed the outputs to see which comparison algorithm would be best, and described each of the algorithms in detail (these videos are linked in our team status report for this week).
Currently, we have a base integration of our systems, but now we need to focus on the use case for the final demo. One idea we were thinking of to help make our game as helpful as possible for the user is to add a side-view for the reference/user dance moves as well, so the user can have a better understanding of what the reference video is doing in a 3D space. Next week, we will be focusing on improvements such as these for the final demo.
Throughout this semester, I had to employ new strategies to learn how to use the tools for our project. One thing that I feel like I haven’t done much in my coursework was learning how to read the documentation on totally new tools that other programmers have created. For example, earlier in the semester, when we were integrating MediaPipe with our system, I had to learn the documentation on the MediaPipe library to get a better understanding of how we can use it for our project. It was a different, more hands-on style of learning that relied heavily on self-guided exploration and practical experimentation rather than traditional classroom instruction. Another important aspect I had to learn during this project was how to effectively work on a subsection of a larger, interconnected system. Our project was divided into different components that could be developed in parallel, but still required constant communication and a shared understanding of the overall design. I learned the importance of interfacing between subsections and how changes in one part of the system can ripple through and impact others. This experience taught me to think not just about my individual tasks, but about how they fit into and affect the broader project as a whole, which is an extremely relevant skill for my career long term.