This week, I focused on improving the integration and user experience of our Unity-based CV dance game, and testing the game overall. I overhauled the game interface by merging the previous dual-avatar system into a single avatar, with the reference motion displayed as a transparent “ghost” overlay for intuitive comparison. I also implemented in-game reference video playback, allowing players to follow along in real time. Additionally, I improved the UI to display scoring metrics more clearly and finalized the logic for warning indicators based on body part similarity thresholds. This involved refining how we visualize pose comparison scores and ensuring error markers appear dynamically based on performance. Our team also coordinated to produce demo videos, highlighting the visual interface upgrades and the functionality of our comparison algorithms for the TA and Professor.
I also dedicated hours this week to tasks including Unity UI restructuring, real-time video rendering, and debugging synchronization issues between the user and reference avatars. Progress is mostly on track, but a few visual polish elements still need refinement as well as a reference playback speed issue. To stay on schedule, I plan to spend additional time finalizing UI. For next week, I aim to finish integrating all comparison algorithm outputs into the UI pipeline, improve performance on lower-end machines, and prepare a final demonstration-ready build.
In terms of new knowledge, I had to dive deep into Unity UI systems, learning how to work with Canvas, RawImage, VideoPlayer, transparency materials, and hierarchical component references. I also read research papers and online resources to understand dance comparison algorithms and the best ways to model human joints from MediaPipe input. Most of my learning came from informal strategies: watching Unity tutorials on YouTube, browsing Stack Overflow, and experimenting directly in the Unity editor. The most challenging part was translating 2D joint data into meaningful 3D avatar motion. This pushed me to study human joint kinematics, including the use of quaternions, Euler angles, and inverse kinematics to replicate realistic movement. I researched how others approached rigging, learned how to apply constraints to joints, and explored interpolation and filtering techniques to smooth noisy input. Through trial-and-error debugging and visualization of bone rotations, I developed a deeper understanding of the math and physics behind body motion, and how to convey fluid, believable movement within the constraints of Unity’s animation system.