Currently, the most significant risks of our project are the machine learning models for the different groups of signs. Specifically, some of the datasets we found to use for training data are not being picked up well by MediaPipe or are not good enough quality, so we are running into some issues with training the model. To mitigate these risks, we are looking for new datasets – particularly for the letters and number signs – and potentially going to be making our own training data for the dynamic signs, as these are the ones with the fewest datasets available online. As for contingency plans, if we are unable to find a good enough dataset that works well with MediaPipe, we might forgo the usage of MediaPipe and create our own CNN for processing the image/video data.
There have not really been any changes to our system design over this past week. One potential change we have been discussing is the grouping of signs over various neural networks, where we might now separate static and dynamic signs rather than dividing signs by the hand shape. This is partially because our static signs are one-handed, with image training data whereas a lot of our dynamic signs are two-handed with video training data. This change was necessary because it makes classification for static signs easier as we can limit the number of hands detected in frame. There aren’t really any costs incurred by this change as we had not yet made models that were separated by hand shape.
Our schedule has also not really changed but we will be allocating some extra time to make the dynamic sign training data since we initially did not anticipate needing to do this.