Shithe’s Status Reports

Status Report 9/21/24

This week I presented the project proposal for our group.  It took some time to memorize the content of the slides as much as possible and prepare for any questions.

Post that, I spent some time this week researching our chosen video game engine, Unity. From that, I’ve been able to break toe game development into a few stages. One stage will be developing an environment for the characters to move in, which will eventually be moved towards being a boxing ring.  The other stage will be developing a character that has dynamically controllable limbs and a body to be used later for control through computer vision. The next stage will be collision detection and damage metrics tied towards that. The last stage will be the game UI development.

Things are currently on schedule but where the schedule will deviate is depending on if I get stuck on any of the stages. Unity development on making an environment will begin this week. If this goes well, I hope to begin on character development as well as a deliverable for next week.

 

Status Report 9/28/24

 

This week I worked on unity development. It started out with a lot of research on beginner methods for making a first person controller on unity. Many of the tutorials online were outdated and included packages which no longer are supported currently which made the process much more difficult than expected, but I was able to find a more recent and in-depth tutorial on some basics for making a first person controller in unity,

I started with downloading a free polygon character package from the unity store and properly importing that into my project as shown below.

After getting more familiar with all of the assets involved in this polygon character, I was moved onto making a simple environment for the character to move on as well as updating the cameras location to be at the head of the character

.

 

I was able to implement basic movements using keyboard controls through C# scripts. That can be shown here.  Everything is on schedule for now. I hope to move onto a custom and more in-depth control set that can take advantage of computer vision for movement to do an early integration test next week.

 

Status Report 10/5/24

 

This week unity progress was continued. I first worked on figuring out how animation in unity worked. This required the creation of some more C# scripts for various portions of animation and then testing to ensure the smoothness of the animation. The animation was done using already existing animations of the free character model. This was done to learn the basic ways of game character animating in Unity before moving on to the more in- depth custom model that will be needed. Some short videos of the progression of this animating can be seen in the following three: 12, 3.

Following that I worked on trying to directly map an example media pipeline pose to the game character. To do this I needed to get a full bone rig onto the character model and then choose out of those the ones that are used by media pipe. After that, using another C# script, an attempt on mapping was done.  Below is an image showing a bit of this.

In that attempt, a few issues were identified that will make this mapping more difficult. One will be the need of converting from the coordinate system of media pipeline which can typically have the origin at something like the top left corner of its image to unity’s which has the center of the environment as its origin (0,0,0). From there the difference in dimensions will need to calculated to convert as well as the difference in character dimensions versus the actual person’s that media pipe captures.

The game logic development is slightly behind schedule as the computer vision integration is being heavily focused on currently, but that is not too much of a worry as that should be the simplest part to integrate into the video game. In the coming week, I hope to research some ways to be able to handle all of these things more easily to get the work done in a fast manner as well as handling making smooth animations between two different poses.

Status Report 10/19/24

A good amount of game logic has been developed. The first thing I worked on was collision detection to be able to tell when a character takes and deals damage to another. Because the medium of damage in a boxing game will be through the fists, I first implemented colliders connected to the middle finger bones of each of hand of the character and expanded them over the full hands as shown below.

From here, I set up a simple cube wall with colliders attached and tested if I could detect when the hands touch the wall. I set it to display in the debug log at the bottom in unity and was able to get that working properly.

From here, I moved onto setting up a simple collider on another character that will be the game AI. A simple health system was set up and attached to the AI character and stand in punching animations were set up to emulate the computer vision punches for the future. With all of this, the player character can now hit the AI and the AI’s health will be displayed lowering on a slider in the game view. A short video of this can be shown here.

From here on, in the upcoming week, I will set up a dynamic collider system for the bodies of the player and the AI so that the game is able to detect the difference between the arms, body, and head being hit and apply different damages amounts depending on what is hit. A slider for player HP will also be implemented. These things will be able to be completed quickly given my experience setting them up for different parts previously. Post these things, I will start to develop a simple AI for the enemy so that it can attack as well as a boxing ring environment. In addition to this, I will work with Taiming to integrate the computer vision controls into the game. The interim demo is in the middle of November. By the next status report, I hope to have an unpolished demo with computer vision controls and developed damage calculation (with sliders) for all parts of the body. We will be on schedule if these things are completed by then.

Status Report 10/26/24

More colliders have been added to the bodies of the characters and increased logic to allow for more dynamic damage detection depending on where the character is hit. This was done through the use of object tags in unity and a script that calculates damage based on object tags.  Images of the change in the the script and new colliders can be seen below.

In addition to that, there was collaboration between Taiming and I in trying to connect the computer vision aspect to the movement of the game character. Some simple issues arose due the non familiarity of either of us with C# scripts which are required to work with Unity but those were overcome quickly. A bigger issue was that the translation of pose coordinates from Mediapipe to connecting with a game character as opposed to sphere objects which have no length resulted in deformed movement of the character.  It is good that we are dealing with this now to remain on schedule for having it fixed by the end of the month. I hope to continue implementing any last bits of game logic as well as continue to work on the integration issue with Taiming over this week.

Status Report 11/2/24

I moved back to focusing on finishing up the full game logic. One thing that i spent a good amount of time on was modifying the threshold required for hit detection so that when multiple hits happen within a very small amount of time, due to a fist not being pulled back yet for example, only one hit is registered. Without correcting this, it could be the case that a single punch could fully take one down from full health to nothing.

We are currently moving into our slack time left behind to get a working demo before the 15th so we are behind schedule as far as that goes but still within the planned extra time that we have left.  By next week, I hope to have made the boxing ring environment as well as added the most remaining simple game logic elements needed for a more fleshed out demo.