This week, I worked on some of the things we had identified to help improve our glove’s performance. I sewed the flex sensors down (so that it is connect to the glove at more than jus a few select intervals). Before, the glove would bend in between connections when our fingers were too straight and the degree of that bend was also contributing to inconsistencies between our training data and our real-time data. Sewing down the glove helped with that a lot and now we are seeing that the glove behaves as expected. The glove is now confused on letters that we know are similar but not on ones that we know should be different. For example, u and w are incredibly distinct letters in the ASL alphabet, but where commonly mis-classified in our previous iteration, but u and r were not even though they are significantly more similar. I believe this is due to the bending issue discussed earlier– now our glove gets confused on r and u but not on u and w. We also discussed using a window of time to see if we could capture the letters that involve motion more accurately. I adjusted the python and arduino scripts to do that as well. We found a few benefits from a window of time after collecting data and testing with a window size of 3 consecutive time instances: the window of time is better at capturing the gestures requiring movement and is also better at capturing the transitions between letters (it knows that those are not to be classified for the most part).
We also met with an ASL user as well as an interpreter to discuss possible improvements to our design and the ethics of our project. One important point that we discussed in both meetings is that we should re-word the scope and use case of our project and add what this can be extended to as “future work.”
I would say we are still on track. We discussed our schedule as a group and determined that our initial schedule with most of our data collection earlier on and iterating later didn’t make sense because iterating on our glove would require getting new data. We have modified our schedule so that we are still in the iterating phase and the bulk of the data collection will happen after we settle on what we think is the best implementation we can achieve in the time we have remaining. This week, I will modify the python and arduino scripts to output audio. I experimented with this a bit this week and found that there are weird syncing issues if only the python script is modified– I believe I need to send the arduino a signal to wait on sending data while the audio is outputting. We will hopefully also decide on our final design and gather data from various people.