This week in preparation for our Design Review Presentation, I worked with Sung and Claire on preparing our slides and finalizing design decisions.
As a part of that, I worked with Claire on choosing the static gestures that we would use for our MVP. Following the advice from our Project Proposal presentation, we choose to modify fingerspelling gestures as many gestures we noticed were very similar or identical. We then remapped the similar gestures into something more distinguishable, and also the dynamic gestures (ie Z) to unique .static gestures. In addition, we also made the unique gestures for popular commands (ie whats the weather).
Furthermore, I worked with Sung on helping to finalize the machine learning implementation we would use. Based on some research on other group’s performances of running OpenPose on Xavier resulting in 17 fps, we found it unlikely that it would run on the Nano, making us pivot to using AWS combined with running OpenCV on Nano. In addition, we prototyped a glove we would use for our OpenCV implementation. The glove was simply a latex glove with a marker marking the joints (ie knuckles). We then did some testing with OpenPose to ensure that the glove would not hinder OpenPose, as well as the effect of distance on OpenPose as well as other factors (ie is face required). We found that the glove did not interfere with OpenPose, that the face is needed for OpenPose to work, and that at roughly around 3 meters OpenPose has difficulty detecting hands (using a 1.2 megapixel camera on a Macbook). We also setup OpenCV and began basic experimenting.
Finally, I continued to make some progress on the Web App, but given the Design Review, I choose to focus more on finalizing design decisions this week, resulting in less progress in the Web App, but with the slack allocated it should be okay.