What did you personally accomplish this week on the project?
Worked on the gesture control algorithm. I implemented a left/right swipe recognition input based on keypoint velocity, but due to OpenPose being fairly slow an the actual keypoints being tracked being fairly noisy, the accuracy isn’t that good. Potential solutions include switching from a velocity-based algorithm to something more positional, such as detecting whether the hand is to the right or left of shoulder, or writing a filter for more stable keypoints –> more stable velocities. Regarding eye-tracking, I managed to extract eye keypoints in screen-space coordinates. Since the camera rig isn’t finished, and the webcam I currently have is fixed, I simply wrote a function to output the signed distance between the average y-position of the eyes and a certain height (in (-1,1) viewport coordinates). This will be input to a feedback algorithm to adjust our camera rig.
Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
I’m supposed to be doing gesture recognition algorithm testing right now (re: schedule), but currently I’ve been strapped for time and not doing that. More refinement needs to be done to the algorithm to make it robust enough for our application. To catch up, I will allocate more time in my schedule for this project.
What deliverables do you hope to complete in the next week?
I do plan to refine the algorithm to be more robust against inaccurate readings from OpenPose, as well as doing some performance fine tuning so that keypoint collection happens faster and the algorithm could be more accurate. The inputs it detects are also pretty limited, will have to discuss more about which gestures we want however. I also don’t have the Jetson currently, and I know I’ve been putting this off literally every week since the second week, but I have yet to build OpenPose on the Jetson.