Posted on

Somya’s Status Report for 3/16/2024

What did you personally accomplish this week on the project? Give files or

photos that demonstrate your progress. Prove to the reader that you put sufficient

effort into the project over the course of the week (12+ hours).

This week I attached all the sensors onto the glove and worked on implementing/testing the circuit. After performing unit tests and discussing an expanded range in our weekly meeting, we determined that op-amp use would be the way to go, and through use of op-amps we managed to expand the voltage output from 0.8V to 2V. 

 

Is your progress on schedule or behind? If you are behind, what actions will be

taken to catch up to the project schedule?

My progress is on  schedule. 

 

What deliverables do you hope to complete in the next week? 

 

Now that we are in the data collection phase, I want to do some more research into how we can clean up the data with noise reduction algorithms. In addition, I would like do some more research on the Bluetooth data transmission so we can more on that in parallel along with the data collection, as well as fabrication of glove 2

Posted on

Ricky’s Status Report for 3/16/2024

This week, I had a hand in multiple elements of our project. I was primarily responsible for the software end things. I finalized the data collection script on the ARDUINO side. I also wrote several scripts to clean up our dataset once we were able to collect the data. I wrote some scripts that allowed for the plotting of the dataset for us to understand how the signings are different from each other. I was also able to test and revise my python scripts that received the ARDUINO data which allowed for successful data collection. I also assisted in glove fabrication. I helped debug issues with the ARDUINO including high source voltage and verifying the ADC. I also assisted Somya in attaching the sensors to the glove. Then, once the glove was ready we spent a decent chunk of time collecting around 30,000 datapoints across the three of us.

My progress is on schedule given the overall pushback of the project.

Over the weekend, I hope to train both NN and SVM using the collected data over the weekend. The plan is to have them trained by Tuesday/Wednesday and present the results to my teammates.

Posted on

Ria’s Status Report for 3/16/2024

This week my main task was to assemble the circuit that would maximize the voltage range output for the flex sensors. We faced the issue where we wanted to use a full 5V range by supplying 5V to each flex sensor, having its voltage output go through an opAmp, and maximizing the range output. The issue we found was that the Arduino Nano BLE Sense Rev2 may be able to supply ~5V (through the computer via USB), but it first of all ranges between 4.5-5.5V and the analog input pins cannot read more than 3.3V because the operating voltage for the Nano is 3.3V. This is an inherent characteristic of the Arduino Nano which is why we cannot reach the full 5V range and we will have to maximize the range to 3.3V. By creating a simple circuit with an operational amplifier, we managed to get a range of 0-2.8V to ensure we don’t max out the ADC. We realistically don’t get near 0 V but getting as close to it as possible diversifies our input as much as possible to give our ML model a good chance at classifying well. We used a simple non inverting amplifier to implement this. 

 

I also soldered wires to our flex sensors to make it easy to connect them to our breadboard. I also cleaned up the circuit we initially made to prevent shorting any connections and making it easier to debug. I also wrote the Arduino script to receive data from the flex sensors and IMU as we were assembling the glove. 

Posted on

Team Status Report for 3/16/2024

The major risk right now is that the data is insufficient to distinguish the various signs for our single-gloved design. To address these issues, we plan on adding additional feature extraction based on our heuristics which could increase the complexity of the data for training. We also have included all 9 DOF of the IMU even though they might not all be needed for feature distinction. This will be something else we look at as part of the risk mitigation. We are also slightly concerned about distinguishing signs that rely less on bending the fingers and more on one finger touching the rest. Our contingency plan is to incorporate 1-2 touch sensors into our design, which if needed, should be a simple addition circuitry-wise. Below are some pictures of the glove fabrication process. 

The only major change is that we have decided to pursue a PCB design for the glove, to simplify the Opamp and Arduino structure on the glove

We have pushed back everything one week, we will shoot to have the first prototype to be done by around Mar. 21. 

Posted on

Ria’s Status Report 3/9/2024

I spent the majority of my time at the beginning of the week preparing and delivering the design review presentation. This was pretty stressful and I think there are a lot of things I would do differently, but it was fun to do a presentation like that for the first time.

My next main task for this period was to make sure we were receiving data from our IMU and formatting that data to look like what we wanted it to feed into our ML model and concatenate with the flex sensor readings. I found that the acceleration readings were a pretty all-encompassing way to capture orientation data. As explained in the previous report, adding up the components in the x, y, and z directions gives us the downward g force, so the breaking down of those components tells us exactly what angle our hand is at. I decided to quantize that data to the nearest .5 to simplify readings but also provide detailed measurements without potential model overfitting.

 

Posted on

Team Status Report for 3/9/2024

        Our team spent our meeting time reviewing each other’s writing on the design report and discussing more details about the content we are including in it. After receiving feedback on our design review slides, we realized that there are a lot of things we still had to churn out, so we effectively spent our time ironing out all of the gaps in our design and transferred that into writing for the report. Things like resistor values, how we will digitize our data, and more detailed testing and validation are a few of the important things we refined. These changes did not incur any additional cost. 

        The most significant risk right now to our project is not meeting deadlines and having good time management. We now have a clear plan of how we want to execute our design, and we have to stay diligent and move forward and meet all of the deadlines. It sounds trivial to just say “we have to stick to our schedule,” but trusting our design process (adjusting as necessary) and being very methodical is what will allow us to do that. From a technical standpoint, a potential risk for our project is our sensor data not giving our ML model enough data. In that case, we would have to order more sensors and wait for those to arrive. We will experiment with our model to find out the outcome of this, but till then we will just be aware of this risk. 

        We have not made any changes to our schedule, and hope to make significant progress on Rapid Prototype I with all our parts finally in one place. 

 

ABET Questions: 

A was written by Somya, B was written by Ria, C was written by Ricky 

Part A: Our project heavily requires the consideration of a global context, because at its heart is the pursuit of better communication, which is a goal that is universal regardless of what language is spoken. There’s some grounding in the fact that even though there are hundreds of languages, there will always be the challenge of making communication between deaf and non-deaf persons as easy as possible. We quickly learned that sign language is also not a universal language—there are somewhere between 138 to 300 variants of sign languages that exist. This information should be factored into the design of our product in the sense that it should be as adaptive and as sensitive as possible. The ultimate end goal would be for any deaf person to slip on these gloves and be able to sign in their own version of sign language, and communicate with someone who may not even speak that language. In essence, it would be just like how Google Translate works, but now with an entire additional layer of recognizing global needs by having that language barrier be extended to not just which country that language is, but the form of the language itself. 

Part B: From our previous analysis of cultural factors, we have adapted our design to be a bit more equitable for users by having the speaker mounted on the glove and not the computer on the final design. We understand that it would be cumbersome to have a deaf speaker have to prop up a laptop or take out a phone when trying to communicate so we want to make that process as seamless as possible. We also want to use a haptic feedback system instead of an LED feedback system for the same reason, maintaining eye contact with someone you are conversing with is something that people should not have to give up just to use our product. 

Part C: Our product doesn’t directly require a heavy consideration of environmental factors, but does so in a few more indirect ways. Right now, since our main goal is to just develop the glove and have it work, the cost is more on the back burner—we are perfectly fine with spending $200+ on all the different components and aren’t too focused on the materials, more so that the materials work. But later down the line, the material choice and cost we do want to more closely consider, because we want our product to be as accessible as possible. This ties into a discussion on natural resources, because we want our product to be easy to manufacture and not require a super complicated and environmentally taxing production process. Our choice of final material for the glove should also be biodegradable and be able to be resistive enough against everyday wear and tear so the user doesn’t need to keep buying them, thus offloading environmental waste. 



Posted on

Somya’s Status Report for 3/9/2024

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I finished the circuit diagram for the sensor wiring (with two sensors displayed as opposed to five for ease of looking at); this schematic is one that I will keep adding to/modifying as our design needs change. 

In addition, I did the mathematical calculations for determining the options for what resistor we want to use in series with each flex sensor. This is quite important to get right because one of the biggest challenges we anticipate after looking at Gesture Glove’s challenges/advice from our TA is flex sensor sensitivity. To address this, we need an output range of values as wide as possible, so if someone signs an ‘a’ versus a ‘b’ the voltage outputs aren’t something like “1.200V” and “1.215V”. As such, we decided that our ideal voltage range would be 1V-4V. This creates the below inequality:

The tricky thing is about having a variable resistor (i.e. the flex sensor), is that the value that satisfies the two equations that can be formed from the above inequality is negative. So, the best thing you can do in practice is form a range of resistor values and play around with multiple resistors within that range to see which ones produce the widest output range. Through my calculations, I found this range to be . As such, I tested five of the most common resistor values within this range: 2.7kΩ, 3.3kΩ, 4.7kΩ, 10kΩ, and 47kΩ. Of course we will test these in the actual breadboard circuit, but I also manually tested them and found that the range reaches maximum of ~0.8V with ~10kΩ of resistance, which isn’t great. So it’s looking like we will need to use an op-amp. I looked at Spectra Symbol’s data sheet, which listed LM358 or LM324 op-amps as suggested. Below is what the circuit diagram for that would look like: . Lastly, I finished all my tasked sections for the Design Report that was due last Friday.

 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is on schedule. 

 

What deliverables do you hope to complete in the next week? 

Now that all of our parts have arrived, I hope to accomplish two tasks in particular. First, I want to do some flex sensor unit testing by building a simple voltage divider circuit and seeing which of the five selected pull-down resistors will give the widest range of V_out. I want to do these units test first so that way if further sensitivity is required by way of an op-amp, I know that before launching into building the entire circuit to accommodate the five flex sensors. Second, after I’ve determined the pull-down resistor through educated trial and error, I would like to have the five flex sensor circuit built by Wednesday. That way we can get to checking if the Arduino code for reading out from the sensors works, and possibly even start the data collection process this week. All our moving parts in preparation for the model training are really close to being done, so hopefully the time we’ve spent preparing everything pays off and data collection goes smoothly. I also want to bring up making a PCB board with the team on Monday’s meeting. 



Posted on

Ricky’s Status Report for 3/9/2024

This week, I primarily worked on the design document. I was responsible for large chunks of the text regarding the machine learning training/testing and the data collection and labeling. I touched upon the use-case requirements, some design trade studies, and big parts of the system implementation and testing. Additionally, I also finalized what would be appropriate for EDA. Specifically, I want to be able to graph the average values of each sensor for each letter and also calculate relevant summary statistics like mean, median, variation, etc.

The software development is on schedule but the actual data collection component is a bit behind. I will just wait for the glove to be developed to catch up. I will keep working on the software until then.

Next week, I will hopefully help assemble the glove and data collection. I will also implement the EDA stated above as well as finish off the last two ML model training modes.

Posted on

Ricky’s Status Report for 2/24/2024

This week, I primarily focused on implementing the script that will train an NN architecture and save the model for future inference use. This implementation has a variety of features including variable choices of input data, cross-validation raining to identify the optimal stepsize, and usage of the wandb package to track training performance through the epochs. I have also implemented the SVM model trainer. This was pretty straightforward with a heavy dependence on the sklearn package. I also tuned up the data collection for an automated collection of 26 letters in one shot. I performed some research into pre-training data analysis based on my “Estimation, Detection, and Learning” class. Based on the research, I added some potential action items to prepare a clean dataset. 

I am on schedule as of right now.

Next week, I hope to put significant work into the design document. In addition, I hope to flesh out a robust testing module, finalize dataset analysis, and begin collecting data with the glove (dependent on the glove’s progress).

Posted on

Team Status Report for 2/24/2024

Right now the most significant risks revolve around getting our parts in time so we can begin construction of the glove and collecting data. We were unfortunately unable to place our orders in time for this week so we will hope to get our parts next week and start assembly of the glove. Meanwhile, development of the major software components (communication, ML) has started.

No major changes have been made to the system. We did buy multiple computing units but we will not choose one of the two until we can test their performance. 

There is a slight pushback in the timeframe to make the first glove but everything else is on schedule.