Category: Uncategorized

This week, we finally finished a working demo of the wearable and Webapp! Joanne and Edward spent several hours debugging and bug squashing in order to properly filter out the incoming finger location data from the wearable. Eventually, they converged on a working system. See video here.

The current issues remaining are that WiFi is fairly slow and negatively affect latency. It isn’t too bad, but it makes our system less real-time than what we wanted. If we were to do this again, we would use BLE or perhaps some other wireless protocol.

Anushka worked on refining the hologram design. She design several version of the hologram and cut them out to see which one gave the best effect. So far, the pyramid with angles that are above 60 degrees and on a big screen work the best. She has also started working on the encasing for the hologram. At this point, we’re not really sure how to improve the visualization without changing it altogether, so we might try bigger angles on smaller screens throughout the week to see what gives the best effect.

We are on schedule to have something for the demo. This week we will work on polishing everything up and making sure things are in order to not blow up at demo day.

This week, we had much to do. Redesigns, bugs, and COVID have pushed us behind schedule. We are on our last slack week next week.

Edward and Joanne have been working on fine-tuning the finger detection and model rotation. The system does not work well on live “real” data and seems to have erratic and random behaviors at times. Hopefully, by the end of this week, we can have a really good version working. We will spend the majority of next week working on this.

Edward and Joanne have also bought black acrylic for the hologram. They attempted the glue the acrylic together, but the glue was not as sticky as we wanted. So, Anushka drafted a new version of the hologram using slots to fit the walls together like a puzzle. Hopefully, we can get this cut and built next week.

We’ve also been working on the final presentation. It’s a good thing we have some time before demo day, but we’re excited to present our findings and what we’ve done so far in the semester.

In good news, the hardware has not bugged out on us yet. It seems fairly reliable so far. We still need to properly sew the board unto the wristband. This should be done by early next week.

Next week will be an extremely busy week for us. Wish us luck!

This week, I did a lot of group work.

I worked with Anushka to set up the Jetson Nano. We had some trouble setting up WiFi, but eventually figured out how to connect to CMU-DEVICE. I plan to set up the MQTT broker on the Jetson in order to improve WiFi latency. Since the broker is closer to the wearable and on the same network, it should be much faster in theory. This weekend, I plan to test this out and see if there is any noticeable speedup in data transmission.

We decided on training a machine learning model to classify the number of fingers on the forearm, since fitting a curve and finding the min and max to classify swipes and pinches was too complicated. So, I worked with Anushka and Joanne to create a SVM classifier to classify the number of fingers. I wrote a training script and trained it on data we had collected a while ago and got about 95% accuracy in cross validation! It seems to perform well on live data, but it isn’t perfect. SVM seems to work well for this task, so we collected some more data and are training it to see if we can get any improvements. The next steps are to improve finger detection given that we know the approximate number of fingers and then improve gesture detection given hat we know the approximate locations of the fingers. I am not too confident that finger detection will be as good as we want, so we may have to deal with that this week when we actually fully implement it. Maybe we need to apply some filtering technique or throw out bad detections.

I also further integrated the wearable with Joanne’s Webapp. The improved finger detection powered by the SVM seemed to be better. Less swipes are classified as pinches which means the Webapp consistently rotates the model.

I am on track for this week, but things are piling up…

This week, I worked on firmware for our wearable. I wrote code for our microcontroller, which is a WiFi-enabled Particle Photon MCU to connect to a public MQTT network and publish sensor data. I actually spent a lot of my time trying to get it to work on campus WiFi. At first, I was unable to get it to connect reliably to CMU-DEVICE, but, after futzing around with it for a couple hours, I tried out another board and it seemed to work fine.
The Photon publishes data formatted like so, in a packed byte array format:
[<4 byte little endian timestamp>, sensor value 1, sensor value 2, sensor value 3]

I hiccup I ran into is that our board cannot get a unix timestamp with millisecond accuracy, so measuring the latency of our data collection will be incredibly off and inaccurate. Might need to figure out a way around this. However, technically, this is only for measuring data for metrics, and not essential to the actual functionality of the device. So, I may need to consider the tradeoffs of taking time to fix this problem or just ignoring it and getting an inaccurate latency measure.

I also worked with Anushka to get a visualization of the sensor data working so that we can understand what kind of data we are dealing with (see video here). We have a set up with three sensors attached to a Photon that publishes data to MQTT. The webpage gets data from MQTT and displays it. I believe visualization will help guide us in what a “finger” looks like in terms of sensor data.

Additionally, I ordered the PCBs earlier this week and they should come next Tuesday (3/1). Next week, I will be working on testing and verifying that the PCBs work as intended. This will probably take up a large chunk of my time next week.

I am on track this week, but as soon as the PCBs come, I might fall behind, since they might not work… I want to get the PCBs verified before Spring Break starts.

Welcome to Carnegie Mellon University: ECE Capstone Projects. This is your first post. Edit or delete it, then start blogging!