Things are going well, last week I was the one to present our final presentation so I prepared a lot of that. I also worked on the wireless implementation and fixed the soldering on the PCB so we can power it with the battery. We are continuing to collect data.
Team Status Report 4/25
The two remaining tasks before demo are wireless integration and to record more trials so the accuracy improves from 92% to 95%. Kat has been focusing on wrapping up the wireless integration since the demo, but we know the glove functions well with the cable connection. It’s possible the wireless implementation will add latency to the data classification, but the delay would be minimal due to the small amount of data we’re sending. More data collection will only improve the model, so there are few risks with continuing.
Software unit testing: Tested the UI/UX with mock results for the entire character set and added buttons (shuffle set, reset, show signs) to improve the user experience.
Hardware unit testing (Teadora): confirmed connectivity/voltage from flex sensors into ADS with a multimeter. Confirmed I2C connectivity (with Kat) by receiving data on the console through cabled connection. Tested multiple conductive pad placements and attachment methods, and confirmed functionality using the suggested libraries for the breakout board and “touched” function.
Overall system test: Tested the full pipeline by collecting data from the pico when the frontend sends a request, sending the data to the model for evaluation, and posting the model’s evaluation to the frontend. We are finding that the accuracy drops significantly when testing the entire system which is what we are currently working to improve.
Teadora’s Status Report 4/25
This past week I worked on the final presentation and started the final poster. The remaining tasks before demo are to complete the wireless integration and record more data so the accuracy improves from 92% to 95%. We plan to meet Monday to review the poster, record more data trials, and hopefully record the final video. I will also start the final report soon.
Nia’s Status Report 4/25
This week, I worked on our final presentation and continued to debug our pipeline testing results. We will continue to work on improving the accuracy this week as well as begin our poster and final report.
Katherine’s Status Report 4/18
Nia and I have been working on the machine learning model’s interfacing with the API. I have implemented the KNN model to work on 5 nearest neighbors, which achieves a testing accuracy of 94%. I also am working still on the implementation of the bluetooth and preparing for the final presentation this week.
Team’s Status Report 4/18
Currently the biggest challenges are with tuning the ML classification model. During the past week we’ve recorded trials to have training data for the model, but we aren’t currently meeting our live accuracy classification requirements. While we expect this will improve with more trials, it also seems likely that there’s bugs in the model-user interface integration.
The hardware integration is finished: the final board is soldered, the wiring is neat and correct, and the battery connection was attached. We’re still using the cable to power the system for right now, since we’re prioritizing getting the classification accuracy higher before integrating the wireless mode.
We adjusted the features the ML model is being trained on to more accurately distinguish the signs, which means we need to rerecord some of our data. Kat and Nia observed that the IMU rotation (pitch, roll, and heading) were helpful in distinguishing some signs. We also removed the translation mode as it would require exceptional model evaluation which we don’t have yet.
Nia’s Status Report 4/18
This past week and the week before, I mainly worked on completing the frontend and integrating it with the pico and backend. Specifically, I created an APIs to deliver the pico’s readings and return the model’s evaluation and confidence value. I also helped Kat with evaluating the ML model and contributed to the team’s overall data collection. We currently have a KNN model that evaluates signs with 94% accuracy. We are continuing to improve the model and debug the results that it returns when testing the entire pipeline.
This week, we began working on the final presentation where we will deliver our results from this past semester.
Through this project, I learned how to implement a Flask API backend as well as the difference/tradeoffs between ML models, specifically KNN and CNN. I mainly did research online and looked at code from past classes and projects to acquire this new knowledge
Teadora’s Status Report 4/18
This week I soldered the lightweight board we’re using for the final demo, then adjusted it after there were some errors. I recorded 33 trials for all 36 characters (1188 trials total) to add data to the ML model, contributing my part to the goal of 100 trials/character. I also wrote part of the final presentation slides and did the design work + copy-writing to communicate my progress since design review.
Weekly extra questions: As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?
A lot of my work this semester involved selecting hardware components to meet the design requirements, then learning how to use previously unfamiliar components. I gained a lot of experience reading and understanding datasheets. I worked a bit with Kat on the Pico integration, so I used online tutorials to brush up on I2C protocols and to understand the ML model Kat and Nia were implementing. One learning strategy I realized I needed to use more was repeatedly reading the datasheets, since the way some of the information was presented was unfamiliar to me and I didn’t always understand it the first time around.
Team’s Status Report 4/4
This week we did our interim demo, we got our glove designed with all the parts mounted onto it via a mini bread-board. We have a through hole breadboard we intend on soldering our parts using to make the product lightweight. This week we will officially add the touch sensors onto the glove as we just completed the testing phase for them.
As for the software, Nia and Kat are working on the data collection script and making it run at the same time as the MicroPython on the Pico. This week we plan on doing data collection with the touch sensors and being able to train the model by the end of the week.
For testing, Katherine made a script that accepted the sensors values and printed their outputs. She calibrated the data collection for optimal accuracy for the flex sensors while bending and straightening the fingers. For our machine learning model, we are outputting the accuracy and loss calculations after training. We are shooting for 95% training accuracy. As of right now we have 3 data vectors per sign, however Katherine and Nia are still working on collecting more. Since Teadora is still adding the touch sensors to the glove, we are manually inputting the touch sensor values for now as they are binary (touched or not touched. The more data we add, the better our accuracy should be. Katherine is implementing a K-NN model and Nia a Neural Network model that we can evaluate both training accuracies on and whichever is better we will use for our final implementation.
Nia’s Status Report 4/4
This week, I completed the data collection script and finalized the Practice page. We began testing but ran into issues with reading from the same port that runs the script. We will research solutions and begin collection data this week. We are a bit behind and may not be able to fine tune our model as extensively, but we will be able to evaluate signs by the end of next week. I will also work on the API endpoints to post to the frontend once we can confidently collect data.