This week was spent with further improvements to the model. Specifically, I spent this week performing some data augmentation in order to improve overall validation accuracy.
The model as it exists currently outputs high confidence intervals for a couple gestures, namely open hand and a fist, when they directly face the camera. For gestures involving a specific number of fingers, the model is less certain and only predicts correctly when the hand is directly facing the camera.
I am still uncertain how to go about improving the accuracy in classifying more complex gestures, but the problem of hand orientation in relation to the camera has a strait forward solution in data augmentation.
Using math similar to that in my former blog post where I explained a 2D rotation, I spent the week creating a script to perform random rotations of the training data about the X, Y, and Z axes instead of just the initial Z axis rotations.
This data augmentation involved projecting the training data points into 3D space at Z=0 and selecting an angle from 0 to 2pi for the Z axis rotation and -pi/4 to pi/4 for the Y and X rotations. After this, the rotations were each done sequentially and the result was project back onto the XY plane by simply removing the Z component.
Next week is shortened for Thanksgiving break, so my ability to work on this project will be reduced. Even so, next week I plan to add two more gesture classes to the OS interface allowing right clicking and scrolling. Time permitting I will also begin collecting tradeoff data for our final report.
Links:
Wikipedia article on 3D rotation matrices
https://en.wikipedia.org/wiki/Rotation_matrix#In_three_dimensions