Things are going well, last week I was the one to present our final presentation so I prepared a lot of that. I also worked on the wireless implementation and fixed the soldering on the PCB so we can power it with the battery. We are continuing to collect data.
Katherine’s Status Report 4/18
Nia and I have been working on the machine learning model’s interfacing with the API. I have implemented the KNN model to work on 5 nearest neighbors, which achieves a testing accuracy of 94%. I also am working still on the implementation of the bluetooth and preparing for the final presentation this week.
Team’s Status Report 4/4
This week we did our interim demo, we got our glove designed with all the parts mounted onto it via a mini bread-board. We have a through hole breadboard we intend on soldering our parts using to make the product lightweight. This week we will officially add the touch sensors onto the glove as we just completed the testing phase for them.
As for the software, Nia and Kat are working on the data collection script and making it run at the same time as the MicroPython on the Pico. This week we plan on doing data collection with the touch sensors and being able to train the model by the end of the week.
For testing, Katherine made a script that accepted the sensors values and printed their outputs. She calibrated the data collection for optimal accuracy for the flex sensors while bending and straightening the fingers. For our machine learning model, we are outputting the accuracy and loss calculations after training. We are shooting for 95% training accuracy. As of right now we have 3 data vectors per sign, however Katherine and Nia are still working on collecting more. Since Teadora is still adding the touch sensors to the glove, we are manually inputting the touch sensor values for now as they are binary (touched or not touched. The more data we add, the better our accuracy should be. Katherine is implementing a K-NN model and Nia a Neural Network model that we can evaluate both training accuracies on and whichever is better we will use for our final implementation.
Kat’s Status Report 4/4
This week I assembled the breadboard with the all our parts mounted onto it and set it up on the glove. For the interim demo I made a script that printed out all the values we were collecting, I helped Teadora test the touch sensors by setting up the library for it and running the script on them as well. Now that our glove is assembled, we are going to start data collection this week. Therefore this weekend I am working on simultaneously running the Micro Python script on the Pico and the data collection script. I also am working on the wireless component so we can run the model without wires when we finish data collection.
Kat’s Status Report 3/28
This week, I worked on processing the ADS and the touch sensors as well as the calibration of the readings coming from the sensors mounted directly to the glove. Now we have a complete set set of readable data from the sensors. I also started work on the wireless transmission of the RBi although I do not think it will be ready by the interim demo. On the hardware side I soldered all the header pins onto our boards like the Raspberry Pi and the touch sensors board so they can be mountable onto the mini breadboard.
This week we have the interim demo so I will work on what I will talk about during it. Nia and I will continue working on the machine learning model as we are thinking of shifting from a neural network to a KNN model.
Kat’s Status Report 3/21
I finished the reading of the data from the flex sensors and IMU. I processed that data into percentages of finger bends with a pretty accurate range. For our project we need at least 2 classifications of bend which I have achieved through calculations right as we receive data. I also made a calibration system for the glove that we can use if needed for different users. Nia and I are also working on the classification of different signs based on binary vectors we created. Teadora and I attached the RPi to a breadboard and can hook up the gloves with its attached flex sensors to it for testing.
This week I will finish the ADS connections so we can read all of the analog flex sensor values. The IMU’s data is a little messed up, I need to finish calculating the position values as they are not correct as of right now. However, it does process motion well and in the correct direction. Nia and I will be working on data collection for our model now that we have started glove assembly.
Team’s Status Report 3/14
This week, Kat did the processing for the Raspberry Pi Pico to receive analog information and send it to the machine learning model. Nia generated a synthetic dataset that she ran the neural network with. Teadora tested the flex sensors and will be continuing to evaluate them and the IMU with the Raspberry Pi Pico this upcoming week.
This week we will make sure we can receive the analog signals properly from the IMU and flex sensors. We will also continue working on the machine learning model and optimize it for the types of data we will be using. We also will complete our work on the data organization such as how the data goes from when it first arrives to on the computer to the machine learning model.
Katherine’s Status Report 3/14
This week I finished the wired connection between the Raspberry Pi Pico so we can read values from the analog signals. Once the ADC adapter comes in we will be able to connect all of the analog signals at once. I plan on finishing the bluetooth processing tomorrow so we can focus on the machine learning model for the rest of the week and time.
Team’s Status Report 3/7
This week we submitted the design report, which required making more decisions about the power supply part of the hardware subsystem. There seems to be some discussion of which battery we will use. Teadora’s proposal is a standard 9V battery with a step down converter to power the RPico and IMU at 5V. Katherine’s proposal involves a 3.7V lithium ion battery to power the RPi Pico and IMU.
This upcoming week we need to test and potentially make some revisions to our design now that our parts have arrived. We will test the parts with what we have so far and determine if we need to make adjustments or not. We also need to order a part to add ADC pins to our RPi Pico, but Katherine has already found the part/s that we can use for that.
Additional questions:
Section A was written by Katherine
Global factors to be considered for our project are for people without access to computers or technology. This is our main global consideration as our product is geared towards people who want to learn, however it would be hard for someone who does not have a computer since our product would work with a computer in pairing. As of right now, our design is dependent on a local machine to run our algorithm and website. As our product is a prototype we are expecting that it would be significantly more accessible with our algorithm and website uploaded to AWS Cloud. Therefore it would be as simple as having the URL, a computer and the glove. However, as our project specifically interfaces with a computer to display results it would change our product greatly to make it accessible to anyone who does not have one.
Section B was written by Teadora
The cultural factors we are considering are beliefs around disability and interpersonal connection. Historically, people with disabilities have been excluded from public life, whether intentionally or through a lack of accessible options. When we were researching existing ASL sensing gloves, a lot of the options focused on recognizing signs and reading them out loud to reduce the need for an interpreter. These design decisions reinforce a cultural belief that speaking out loud is a better form of communication. However, they don’t respond to the reality that sign languages are their own complete languages, separate from spoken language. As a note from the Virginian community college Germana states, “The sentence “I see a big orange cat” would be signed as follows:CAT, ORANGE, BIG, I SEE [1]. In situations where quick and accurate communication with someone who uses ASL is needed, the existing gloves won’t suffice, and in some ways continue to exclude the deaf community from public life. Our design is based in the belief that learning ASL is a better solution because it connects with an existing language and culture. Our project emphasizes specific users: people who are interested in learning ASL, perhaps to communicate with friends or because they’re losing some hearing themselves. Another cultural factor is that ASL has started to increase in cultural capital. It’s seen as impressive for someone to know ASL, and this is emphasized by the inclusion of ASL interpreters in popular entertainment, like concerts and TV shows. Our design recognizes that communicating in ASL is increasingly necessary and desirable, not just to the deaf community but to the broader American public.
Section C was written by Nia
The environmental factors for our device mainly relate to the materials used in the hardware and the energy consumption of the device. Because our design uses electronic components such as the Raspberry Pi Pico, an IMU sensor, wiring, and a battery power supply, it contributes to the broader issue of electronic waste. Electronic devices often contain metals, plastics, and batteries that can be harmful to the environment if they are not disposed of properly. When selecting components for our prototype, we considered using available and reusable parts so that they can be repurposed in other projects rather than immediately discarded.
Another environmental consideration involves the type of battery used to power the system. The team discussed using either a standard 9V battery with or a 3.3V lithium-ion battery. While 9V batteries are easy to obtain, they are often disposable and may contribute more waste if replaced frequently. Rechargeable lithium-ion batteries, on the other hand, can be reused many times and generally produce less battery waste over the lifetime of the device. However, lithium-ion batteries must also be handled and recycled properly because they contain materials that can be hazardous if they end up in landfills [2].
[1] “Provided by ASL Grammar Guide The Academic Center for Excellence 1 ASL Grammar Guide,” 2023. Available: https://germanna.edu/sites/default/files/2023-07/ASL%20Grammar%20Guide%20%28edit%207-24-23%29.pdf
[2] Vermont Department of Environmental Conservation. Lithium-Based Battery Management Fact Sheet. 2020. Available: https://dec.vermont.gov/sites/dec/files/wmp/SolidWaste/Documents/lithium-basedBatteryManagementFactSheet.pdf
Katherine’s Status Report 3/7
This week I worked on the design document and researching the machine learning model. I have begun with the serial processing code however I need the microcontroller to test it. As the microcontroller arrived this week I will be able to work on that and see how it works. I made some revisions to our design as we do not have enough pins on our microcontroller for our analog signals. Therefore I am ordering an ADS that will give us extra channels for our analog signals, I will discuss with my team on Monday before placing the order. I found one that adds 4-channels and that was original plan, however as I do research I discovered that a lot of people end up using touch sensors so I am wondering if implementing an 8 extra ADC pins would be needed.