Posted on

Stephanie’s Status Report for 11/6

This week, I worked on implementing new code for data parsing and model training and testing since we have decided to take in a window of data (three data points).  We still decided to use random forest for our classifier with the new data because after running through some default models, it’s still the model with best accuracy. We are also considering neural network since it can achieve >90% accuracy after hyperparameter tuning and the high dimensionality data may lead to better performance with neural network.

I also performed some data analysis with older sets of data on letters that the classifiers got confused on but were not similar in poses. I noticed two things that may help us in the future. First was that there were some big outliers in the data we collected. Usually, flex sensors’ readings are below 50~300 (for the letters I analyzed), but there were a few large outliers like 27600 that may have occurred due to sensor errors or data stream not being read correctly. While data are normalized before used for training, I did not explicitly implement any steps to get rid of outliers, so that’s something I will add on for future model training. Another discovery I made is that the flex sensors had similar values when they shouldn’t have, e.g. a curved finger vs. a straightened finger have similar range of values. This possibly led to the classifier being confused. This may have occurred because our sensors were not that tightly sewn to the glove. After the preliminary testing done for our newest set of data, we found that the classifier had much less trouble in identifying dissimilar letters that it was previously confused with.

We met with two ASL users too this week, one of whom is an interpreter. We talked about ethics and the extent of our project and got some helpful advice on addressing the scope and the users of our work.

We have made changes to our schedule since we realized that it doesn’t make sense to bulk data collection  until our glove build and data collection process is finalized and adjustments are made based on feedback. Hence we are still on schedule. In the next week, we will be collecting more data and I will work on training/testing with other models since our newest classifier still gets confused on similar letters.

 

Posted on

Rachel’s Status Report for 11/6

This week, I worked on some of the things we had identified to help improve our glove’s performance. I sewed the flex sensors down (so that it is connect to the glove at more than jus a few select intervals). Before, the glove would bend in between connections when our fingers were too straight and the degree of that bend was also contributing to inconsistencies between our training data and our real-time data. Sewing down the glove helped with that a lot and now we are seeing that the glove behaves as expected. The glove is now confused on letters that we know are similar but not on ones that we know should be different. For example, u and w are incredibly distinct letters in the ASL alphabet, but where commonly mis-classified in our previous iteration, but u and r were not even though they are significantly more similar. I believe this is due to the bending issue discussed earlier– now our glove gets confused on r and u but not on u and w. We also discussed using a window of time to see if we could capture the letters that involve motion more accurately. I adjusted the python and arduino scripts to do that as well. We found a few benefits from a window of time after collecting data and testing with a window size of 3 consecutive time instances: the window of time is better at capturing the gestures requiring movement and is also better at capturing the transitions between letters (it knows that those are not to be classified for the most part).

We also met with an ASL user as well as an interpreter to discuss possible improvements to our design and the ethics of our project. One important point that we discussed in both meetings is that we should re-word the scope and use case of our project and add what this can be extended to as “future work.”

I would say we are still on track. We discussed our schedule as a group and determined that our initial schedule with most of our data collection earlier on and iterating later didn’t make sense because iterating on our glove would require getting new data. We have modified our schedule so that we are still in the iterating phase and the bulk of the data collection will happen after we settle on what we think is the best implementation we can achieve in the time we have remaining. This week, I will modify the python and arduino scripts to output audio. I experimented with this a bit this week and found that there are weird syncing issues if only the python script is modified– I believe I need to send the arduino a signal to wait on sending data while the audio is outputting. We will hopefully also decide on our final design and gather data from various people.