We had interim demos this week. My group demoed the functional portions of our project to course TAs on Monday and Professors on Wednesday, are received positive feedback, as well as suggestions for UI to add and potential training and dataset augmentations.
Personally, I spent this week improving the gesture recognition model, adding robustness to the predictions made by the model by introducing random rotations to the training data. This is done by initializing a random angle theta and constructing a 2×2 rotation matrix
[ cos theta, –sin theta ]
[ sin theta, cos theta]
That is then multiplied by the 2×21 matrix containing the landmark coordinates. These transforms resulted in much better accuracy recognizing the click gesture when user hands were not directly vertical.
I will spend next week further refining the model. Our current implementation is very accurate classifying open hands and closed fists, but is still struggling some with hand representations of number and the other and signs.