Category: Team Status Updates

This week, we worked on finalizing our code for the project. Rosina and Saumya worked on expanding the code for the IMU to the z-axis (decoupling x and z and creating separate Kalman filters for each), and flipping the direction of the z-axis. Sarah practiced and presented our final presentation slides and took the lead on making the poster, while Saumya and Rosina helped write the different sections of the software portion.

One interesting problem we solved was the failsafe triggering. We tried to make the failsafe interrupt the IMU to avoid as much drift as possible and “freeze” the mouse in place when the failsafe was let go. No considerable changes have occurred to our design, and we are seemingly on track with the rest of the project. There are no risks to us presenting what we planned to present in the demo.

To answer the ABET question:

Latency: To test our latency requirement, we manually started a timer while at the same time triggered one of the sensors on the glove. In this way, starting the timer would be manual, and the timer would stop when the gesture propagated through. We achieved an average of 40ms for keystrokes.

We noticed a ton of latency (on the order of seconds) when testing our mouse movements, so we tried to reduce the amount of calls to pyautogui and reduce the amount of computation. Specifically, if the mouse position did not change between IMU data arrivals, we didn’t call pyautogui. This improved latency of the mouse to below 1s.

Weight: Unfortunately, we were unable to reach our weight use case requirement, weighing in at ~20g over our budgeted weight. We prioritized battery life as it is more inconvenient for the user to recharge their device multiple times during a session. Additionally, 20g is not very significant in the realm of human detection. Because of this immense increase in battery life, we decided this trade off was well worth it.

Accuracy: To test accuracy, we conducted a user study in which we guided each user through all the available gestures on the glove. Participants were asked to carry out each gesture and we recorded whether this gesture produced the expected shortcut. We were able to achieve a 100% accuracy after many iterations of sensor positioning.

Wireless Range: To test wireless range, we slowly incremented the distance between the glove and the device it was paired to. For each ½ a foot, we triggered one of the sensors and recorded whether the desired gesture was produced. We found that the final distance that triggered a gesture was 3.05 meters (10ft).

Battery Life: To test battery life, we connected our product to a battery of known voltage, current, and battery capacity. We continuously sent out packages over Bluetooth until this battery ran out. Since we are using Bluetooth Low Energy (BLE), we were able to achieve a battery life of over 12 hours for a 5V, 2.1A, 10,000mAh portable charger.

This week, we started integrating our hardware with the glove and thresholding sensors now that they are in their final positions. The software portion of our team worked extensively on IMU mouse movement algorithm. Our product is now able to move the mouse left and right, and we plan to extend this code to vertical movement along the z-axis in the early half of this week. We also worked this past week on our slides for the final presentation.

No major changes are being made to the device, but the IMU code may impose some limitations on how long the mouse can be used before it needs to recalibrate. The Bluetooth protocol was also modified slightly to send IMU data using a bytearray rather than an array of floats due to compatibility issues with the ESP bluetooth library we were using.

This week more work on was done with the bluetooth protocol, glove, mapping keystrokes, and IMU. Significant risks we are currently facing is the size and usability of our device. We are entering the testing phase of our project now that the physical components are being placed on the glove. The components are a bit bigger than expected. This might interfere with the accessibility of our product because it may not be as comfortable, however, it is necessary to meet our other requirements. No other changes are being made to the device.

This week we all focused on getting our parts done for the interim demo. Saumya had the bluetooth portion, Sarah had the sensors and hardware, and Rosina had the IMU. We are currently working on improving our respective parts and integrating them together. As for changes to the design, we went back to the bluetooth protocol instead of Wifi. We are also making new changes to our PCB board to add in resistors and make it smaller in size. Our costs are still mitigated and was necessary in terms of creating a more accessible device. Some risks we are currently facing is making sure we don’t use up too much memory since only limited RAM is available on the ESP32. After we get out parts working we will focus on minimizing the memory.

This week, we worked on our ethics assignment and thought about ways in which our product would affect users in an ethical manner. One of the cases we thought about specifically was the situation in which someone could control someone else’s device without their consent. We thought about ways in which we could combat this, including limiting information collected from the user as well as security measures to make sure the user knows exactly which keystrokes do what and the risks that come with using the device. Additionally, we received our IMU this week and soldered jumper cables onto it. We will work next week on creating a demonstration for the class as well as working with the IMU to interface it with the ESP32, specifically implementing communication protocols as well as figuring out how to access information from the device. Some risks that could jeopardize the project are, because we received the IMU somewhat later in the semester due to shipping complications, we need to start interfacing and familiarizing ourselves with the software and calibration techniques that come with the device. No considerable changes have occurred to our design, and we are seemingly on track with the rest of the project.

This week, we planned out our code for when we receive our final set of sensors. Sarah also finished routing and milling our PCB. In terms of the risks that our project currently faces, we still haven’t received our IMU, so we can’t start fully coding that part yet until we can figure out and play around with the IMU’s calibration and sensor outputs. We are also facing issues with the PCB – specifically, Techspark’s PCB mill was not accurate enough for our small PCB, and we identified shorts between the traces on the board. We are planning on either using a vector board or ordering our PCB, but with the increased shipping time, we are a little bit worried about how this will affect the timeline of our project. We plan to make our decision about the board by Monday when we can place orders, and update our timeline accordingly if necessary. 

This week was our budgeted slack week as per the Gantt chart. In terms of the risks that our project currently faces, we are a little worried about the shipping time for the IMU and the printing time for our PCB. We ordered the IMU right before break, and we are hoping that it’ll get delivered by the time we get back. As for the PCB, since Techspark was closed this week for break, we couldn’t get a head start on milling our PCB. If we can’t get the PCB milled, we can test with a breadboard until we get the PCB. If the IMU doesn’t arrive, we’ll need to move things around a little to budget more time to play around and code for the IMU. We haven’t made any changes to the design in the past week except deciding to mill the PCB ourselves instead of getting it outsourced.

This week, we received some of our parts and began testing our board and the sensors. We focused on thresholding our force sensitive resistors to determine good potential values for what would count as a button press. We also looked into BLE implementations for the specific board we selected. The most significant risks for our project now are that two of the pins on our board are shorted. We’re planning to order another board this week and experiment with the current one, but we’re hoping that this integrity issue only applies to the board we received. Besides that, our sensors seem to be working fine (except for one that was a little faulty, but we ordered one more than we needed so we should be alright). In addition to this, we also refined our slides for the design presentation, and we began working on the design review report.

When developing our solution we used the following engineering principles: economy of mechanism, top-down design, and design for contingency. For economy of mechanism, we wanted our design to be as simple as possible while still achieving our design requirements so that there would be as little room for error as possible. We also wanted our design to be simple and intuitive to use. When designing, we chose a top-down approach, focusing on the bigger parts of the design and our end goals and then backtracking all the way down to the low level code and hardware wiring that would help us achieve our ultimate goal. Finally, we designed a lot for contingencies, in case any of our tests failed. We came up with proactive risk mitigation strategies and backup plans so we’d be prepared.

This week, we did lots of research both on the software and hardware sides of the design. We’re starting to work with the Python libraries required for the mouse tracking part of our project, as well as designing power structures and schematics for the hardware side of the project. The most significant risks at this point involve shipping time and integrity of the parts we ordered. Since we ordered our parts mid-last week, we hope they will arrive in time for us to work on this upcoming week. To mitigate our shipping time, we have chosen to source some of our parts from Amazon, but the integrity of our parts may be compromised in this. To fix this issue, we ordered more than enough parts necessary, so in the event some are faulty, we have backups. We decided to switch from using a nucleo board to using an ESP32 dev board, because of it’s multitude of ADC channels, bluetooth capability, but mainly because of its size. The new board we’ve chosen is a bit over 2 inches, which would comfortably fit in the palm of one’s hand. No other significant changes were made to our design, aside from the fact that we are still in process of figuring out the power structure for our design. We are still largely on schedule, and are awaiting the parts to ship.