I started this week by continuing to work on the hand tracking pipeline—mostly adding in two-handed tracking abilities. My main focus of the week was working with pianists on Wednesday. The hand tracking ran smoothly with the pianists and we were able to record some footage (linked here). I made a key discovery during the session—hand tension isn’t determined by set deviation in either direction. Tension occurs when the wrist is held in one position for too long. So, I’ll be changing the pipeline to look for changes in wrist angle instead of specifically looking for ulnar/radial deviation. If the wrist remains at the same angle for too long, the pipeline will signal for a buzz.
There are no new issues with the pipeline. I still need to integrate a way to tell which direction the wrist is deviated—however, since we’re now tracking wrist angle changes, this is more of a “nice to have” rather than a necessary component.
Despite the change, I’m still on schedule. For next week, I’ll focus on continuing to beautify the code I have as well as looking into how to port Mediapipe to Pytorch. Jessie has started looking more into the KV260 and discovered that TFLite, the framework Mediapipe is based on, isn’t supported by the system. We’ll need to convert it to Pytorch or Tensorflow instead. I found more online resources outlining how to port to Pytorch so I’ll start looking there first. Finally, we’ll be putting in the order for our camera stand next week, so I’ll also begin assembly for that once it arrives.