Tag: status report

Last week, I helped making the final presentation slides and worked on generating graphs for our machine learning models to compare the trade-offs and accuracies from different rounds of testing for our final presentation.

In this week, we’re mostly making adjustments to the physical components of the glove, so there isn’t much to do on the software end. I wrote a new script for data training since the format of the data we collect after on will be different from the previous format as we added more sensors. With this, we can get started right away with model training after data collection.

Next week, we’ll be meeting up to collect data after the sensors are all tightly attached and finish up our final report and video. I have also contacted others for data collection.

 

This week I worked on integrating the letter-specific models with our old classifier. The original classifier is responsible for identifying the input data as one of the 26 letters. If the predicted letter is in the list of similar letters that the original classifier easily gets confused on, notably R, U, V, W and M, N, S, T, E, the data will then be passed to one of the smaller models for further classification. This gave slightly better results and was able to make more correct classification.

I performed further data analysis to see if there’s any other way I can improve the model. I did find some data contained invalid values (negative values from flex sensors) that my old outlier removal function did not pick up, hence I was able to refine that to get better data. The sensor values for U and V are rather similar so it could be hard to have a classifier to distinguish between these two. R’s range of values is quite different from the rest of the letters for the middle finger, hence to improve future models for this specific group of letters, I may only train with data of important features, e.g., the flex sensor values.

During this, I also found the ring finger’s flex sensor value for W (finger is straight) is quite similar to that of R,U, and V (finger is bent). Upon further testing with the glove, I found that the flex sensor on the ring finger is not working as intended and is only giving about the same value no matter the bending angle, so we are looking to replace that sensor.

I believe we are still on schedule. In the next week, I’ll be collecting new data after the flex sensor has been fixed.  I’ll also be doing some real time testing and experimenting with new ideas of the smaller model (e.g., training on only important features, changing hyperparameter to produce models with less accuracy on the collected data but more generalizability). After that, we’ll be working on our final presentation and report.

This week, I worked on implementing new code for data parsing and model training and testing since we have decided to take in a window of data (three data points).  We still decided to use random forest for our classifier with the new data because after running through some default models, it’s still the model with best accuracy. We are also considering neural network since it can achieve >90% accuracy after hyperparameter tuning and the high dimensionality data may lead to better performance with neural network.

I also performed some data analysis with older sets of data on letters that the classifiers got confused on but were not similar in poses. I noticed two things that may help us in the future. First was that there were some big outliers in the data we collected. Usually, flex sensors’ readings are below 50~300 (for the letters I analyzed), but there were a few large outliers like 27600 that may have occurred due to sensor errors or data stream not being read correctly. While data are normalized before used for training, I did not explicitly implement any steps to get rid of outliers, so that’s something I will add on for future model training. Another discovery I made is that the flex sensors had similar values when they shouldn’t have, e.g. a curved finger vs. a straightened finger have similar range of values. This possibly led to the classifier being confused. This may have occurred because our sensors were not that tightly sewn to the glove. After the preliminary testing done for our newest set of data, we found that the classifier had much less trouble in identifying dissimilar letters that it was previously confused with.

We met with two ASL users too this week, one of whom is an interpreter. We talked about ethics and the extent of our project and got some helpful advice on addressing the scope and the users of our work.

We have made changes to our schedule since we realized that it doesn’t make sense to bulk data collection  until our glove build and data collection process is finalized and adjustments are made based on feedback. Hence we are still on schedule. In the next week, we will be collecting more data and I will work on training/testing with other models since our newest classifier still gets confused on similar letters.

 

In this week, I worked on researching on why the neural network’s accuracy increased with the added labels of “space” and “transitions”. To compare the accuracies of different data sets, I compiled three different sets of data: one with all the labels, one with all the letters and the space labels, and one with only letter labels. To see if the differences in data did affect the overall accuracy, I performed hyperparameter tuning when training each data set’s model to find the best performing accuracies. The results showed that best tuned models were able to reach 93%+ accuracy for all the data sets.  (The accuracy also varies by runs due to random initialization in the model)

In conclusion, the added data labels didn’t actually improve the accuracy of the model. Since neural networks can have various amount of hidden layers and activation function, it’s likely that we just did not find the optimal parameters of the model for a particular dataset.

Furthermore, we found that our first set of data were mostly taken with the hand facing one direction. This may significantly impair our model performance. This was further proved when I tested the model trained on first set of data. On our two newest sets of data, the model only showed 4% and 21% accuracy. Hence it’s likely that we will not be using this set of data in the future.

We also adjusted the IMU on the glove this week and just got a new set of data. Preliminary tests on this dataset showed high accuracy (98%), but model trained on second data set only had a 50% accuracy on this set, which signifies this is a large change to the overall data. We’ll be doing more real time testing next week. I’ll also be working on making some data comparison with our older data to see why the old model did not perform as well in real time testing.

I would say we are a little behind schedule in terms of gathering data as we are still contacting people who use ASL. But we’re doing well with data training/testing and model analyzing.

 

 

There has been a change of plan from what I wanted to do from last two weeks. My original plan was to collect more data for model training, however, our glove needed more fabrication work to ensure the sensors are well attached. We also plan to enhance the glove’s data collection process. Our first set of data (done by Rachel) had to be collected by pressing buttons to determine the time duration of when the data will be read in. So for this week, we are trying to integrate real-time data collection and interpretation. More data is collected for hand gestures in-between each letter gesture (these gestures are mostly random since they are just transitions from one letter to another). A ‘space’ letter is added in case the models can not categorize the ‘random’ gestures well. First round of testing shows promising result. With the random forest model, which has been the model with highest accuracy so far, the accuracy for recognizing these two new labels are quite high.

I also found that with these two new labels added, the neural network accuracy have increased by 10%. This is an interesting finding and I plan to look further into why this is the case. Before this, I have done much tweaking to the model and validating with different hyperparameters, but the network’s accuracy seemed to be capped around 80%, however, with this new dataset, its accuracy reached around 88% and that is something I would like to find out why to help our future classification works.

I would say we are still on schedule since we planned a lot of time for the software implementation. As for next week, I’ll be looking more into the models and working on make our data collection work more smoothly, and if possible, starting to collect data from others.