This week, I focused on improving the natural movement of the character in our CV-based dance coach. By manually adjusting the targets that each joint should track, including arms, legs, and neck, I was able to refine the character’s motion to make it feel more fluid and realistic. The attached video demonstrates these improvements, showing how the avatar’s motion now closely mirrors user input in a more natural way given 3 different dance test poses to move to/from. Additionally, I worked on optimizing our UDP network communication, ensuring that packet transmission and reception are stable with good throughput. One major challenge I encountered is normalizing the coordinate data from MediaPipe’s OpenCV-based pose tracking to Unity’s avatar system, as they operate on different scales. To better understand how real-time feedback should be structured, I also consulted 3 friends who enjoy TikTok dancing and learning these trendy dances on the fly. Based on their input, I confirmed that a Just Dance-style single-frame feedback system is intuitive, fun, and easy to engage with, making it an ideal approach for our project, so I’ve decided with our teammates that this should be what we look for in our design requirements as well.
Currently, our progress is on track with our proposed schedule, as we are actively working on synchronizing user input with the reference video using CV techniques. We have also taken feedback from our Design Proposal Presentation this week to incorporate a more refined design requirement for the feedback part of our system. Moving forward, my next major focus will be on normalizing the MediaPipe pose estimation coordinates to correctly align with Unity’s character rig. This is a crucial step in ensuring accurate comparisons between the user’s movements and the reference video. Additionally, this will augment our upcoming tasks of making a real-time feedback system. If any delays arise on my part, I will dedicate additional time to debugging and adjusting the coordinate mappings to keep us aligned with our project timeline.