This week, I worked with Aishwarya to test the initial model we have for static signs (1-finger, 2-finger, 3-finger, fist, etc) and discovered some discrepancies in the training data for the following signs: 3, e, f, m, n, q, t. As a result, I (along with my other two group members) created some additional training data for these signs in order to retrain the model to detect the correct version of them.
Additionally, I worked on the normal testing data that I had assigned for this week (letters a-m, numbers 0-4), in accordance with the group schedule. I also began brainstorming ways to account for choosing the highest model prediction, as each of the possibilities in our model add up to 100 rather than being out of 100 individually. This means that we cannot specify a specific range of prediction values for deciding the best one, as we previously thought, but instead will be grouping any unrecognized movements by the user into an “other” class to ensure that the near real-time prediction is accurate.
Furthermore, for the interim demo, we are brainstorming some final aspects of the webapp such as the most intuitive way to display feedback to the user and having easy to understand instructional materials. As part of the instructional materials, I created text blurbs for all 51 signs that specify how to do each sign (along with the video) as well as certain facts about the sign (i.e. “help” is a directional sign where directing it outwards indicates giving help while directing it towards yourself indicates needing/receiving help).
At the moment, our project is on schedule with the exception that we are beginning integration a week early and that we have to account for some extra time to make the training data from this week.
As for next week, I plan to continue making testing/training data, work with Valeria to integrate the instructional materials into the webapp, and prepare for the interim demo with the rest of my group.