Lilly’s Status Report for 4/19/2025

Things done this week:

  1. Tweaked angle calculation code more to account for the weirdness at the 90 degree angle mark. Tried a bunch of different calculation methods again, ultimately moved to a smaller Kalman filter that I could tune properly instead of a black-boxed Arduino library filter. Tuned the filter to make the outputs less noisy while still getting to the extreme angles.
  2. Debugged calibration commands between server and Pi. Should be finalized now.
  3. Finished hat assembly, for real this time (made some straps with buttons to hold the components in place).
  4. Started video testing with Kinovea to get a more quantitative sense of how to fine-tune the filter. Got a 10 min and 30 min sample (was going for an hour but I forgot to turn automatic sleep off on my computer…). Did the testing with 2 different people so far.
  5. Worked on the final presentation slides.

 

Progress?

Still in testing/benchmarking/tweaking mode since debugging stuff with the angle calculation took a little longer. The results from the first 2 testing sessions should be out today but I thought I’d type up this status report first.

 

Deliverables for next week:

  1. Set up “demo” and “real use-case” modes
  2. Finish testing over longer (1 hr) periods of time, and further tweak filters as needed.
  3. Finish final documents… 

 

Learning?

I got a lot more familiar with the Arduino IDE and Python programming to get all the coding done for this part of the project. Luckily there were a lot of relevant Arduino libraries (particularly for bluetooth control and sensor drivers) so I was able to modify much of the example code to make it work for this application, along with adding the extra things I needed. I also learned about different angle estimation methods, including complementary, Kalman filters, and just doing trig on the acceleration vectors. This involved reading a lot of forum posts about the filters, and papers comparing the methods/explaining their disadvantages and advantages. Another thing I learned was how to use HTTP post and get requests, since I had never done that before and we needed a way to send data to and from the server. I learned this from my teammates, googling, and guess-and-checking with print statements to debug. One tool that I learned about for this project was Kinovea, which isn’t actually great for automatically tracking angles but that’s fine since I’m just going to manually measure them using the software anyway. The docs for Kinovea and the user interface are fine so I didn’t have to do much external research to figure out how to use the tool.

 

No images since I haven’t processed/analyzed my data yet :,,)

Here is a snippet of a testing session to demonstrate the setup:  https://drive.google.com/file/d/1BQTXiCdb4EkZl6gPwkAzyboGbDKy8ZFB/view?usp=sharing

Team Status Report for 4/12/2025:

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

  1. For the seat module, one risk is the results of our current lean-detection algorithm being overly variable from person to person, due to different weights or ways of sitting on the seat. This risk is being managed by testing with many different people to see what the variability looks like and refine our algorithms accordingly.
  2. In the neck module, one risk is the accumulation of errors over a longer work period. So far, the angle calculation has been tested by moving the sensor around over a short period of time, and keeping the sensor stationary (on a breadboard) over a long period of time to see if the angle drifts up or down. These tests have yielded positive results, but the system has not yet been tested on a moving user over a long period, which is what we will do this week. While the sensors have been calibrated for drift on a breadboard, this error is correlated with temperature so it may change as a user wears it for a long time. This risk is just being managed by testing. If we find that the angle calculations are indeed getting worse, we can implement a more complicated recalibration procedure (where the sensor offsets are actually recalculated). However, I doubt that temperature changes will be that problematic since we’ve worked hard to give the sensors good airflow and distance from the user’s skin.
  3. Within the browser extension, one risk is the security concerns associated with running third-party code to display the graphs. This risk is being mitigated by sandboxing the code that uses this third-party code!

 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

  1. Decided to handle all of the user<->neck system interaction on the Pi instead of offloading that computation to the ESP32 as originally planned. This was mostly a result of wanting to speed up debugging/testing time, as compiling python scripts on the Pi is much faster than compiling using the Arduino IDE, and this allows us to test the neck system in a wireless mode. The original plan to do more on the ESP32 was in place because I thought I would have to completely redo the sensor offsets each time the user wanted to recalibrate, but this turned out to be unnecessary. This change also reduces the amount of power consumption on the ESP32’s end and reduces the latency between user calibration command -> change in reported angles, so it’s overall a win. No extra costs incurred.
  2. Changed the plan for laying out all the components on the hat. Originally the goal was to minimize the wiring between components, but this meant the hat’s weight distribution was awfully lopsided so I just worked on making the wiring neater. I also switched to using velcro to attach the pockets holding the parts instead of a more permanent attachment (sewing or superglue) to allow for some flexibility if I need to move something. No additional costs since I already had velcro.
  3. Using a battery instead of a wall plug to power the circuitry for the sensor mat (not including the Pi). No additional costs as we also had batteries lying around. 

 

Provide an updated schedule if changes have occurred.

It seems we have arrived at the “slack time” of our original schedule, but we are in fact still in testing mode.

 

System Validation:

The individual component tests (mostly) concerning accuracy (reported value vs. ground truth value) of each of our systems have been discussed in our own status reports, but for the overall project the main thing we’d want to see is all of our alerts working in tandem and being filtered/combined appropriately to avoid spamming the user. For example, we’d want to make sure that if we have multiple system alerts (say, a low blink rate + an unwanted lean + a large neck angle) triggered around the same time, we don’t miss one of them or send a bunch at once that cannot be read properly. We also need to test the latency of these alerts when everything is running together, which we can do by triggering each of the alert conditions and recording how long it takes for the extension to provide an alert about it.

Lilly’s Status Report for 4/12/2025

Items done this week:

  1. Sewed all the pockets for the parts on the hat, finalized the layout of all the components, and soldered ESP32 to IMU so they can be inserted into the hat without terrible wires sticking out.
  2. Wrote/tested recalibration code using server<->Pi communication, and synchronized this with requests from the browser extension. I decided to just handle all the recalibration on the Pi since I realized the process doesn’t have to be that complicated to work properly.
  3. Implemented simple low pass filtering (just a moving average) to kill some of the bumpiness in the angle data. Still using the Kalman filtering/sensor fusion algorithm from the interim demo. Seems to work well enough (main thing is it doesn’t drift up/down at rest) but I need to test with a camera instead of just eyeballing to see how accurate the “real-time” angles are.
  4. Tweaked code for sending alerts to avoid over-alerting the user.
  5. Tested and debugged wireless setup for hat system.
  6. Roughly tested with a person wearing the hat.

Progress

Ready to start verifying the angle calculations at this point now that everything is attached to the hat and the wireless mode is fixed, so I’m not too behind schedule. One unfortunate thing is that the JST connectors I ordered ended up not being the right size so I will have to do some more soldering on Monday to extend the connector on the battery. This is not too much of a roadblock for testing, since there’s a convenient little flap on the hat that the battery can be tucked into, close to the ESP32. And, the soldering job will be quick anyway. Overall I don’t think there will be an issue with finishing the final tweaks + testing of this subsystem in time for the final demo.

Deliverables for next week

  1. Make components more secure on the hat + solder extension wires to battery connectors
  2. Test calculated angles with a camera
  3. Configure a demo and “real” mode.

Verification

  1. User testing with the hat – have someone wear the hat and do some work at a computer, record from side view (~10 min), calibrate ~90 degrees (upright) as starting position, have a screen with the current angles being printed in the frame, take a sample of ~20 frames from the video, use an image analysis tool (e.g. Kinovea) to get a ground truth angle to compare the “real-time” data with. Test “passes” if the ground truth angle and calculated angles are within 5 degrees in all frames. I also want to repeat this test on a user working for an hour, and take an even sampling of 20 frames across this hour to see if the accuracy gets worse over time.

Video of initial testing (this still with the jumper wires, forgot to re-record with soldered version):  https://drive.google.com/file/d/1_j-dwfMfuiTcbsXzWj6ul1kb8jZTK9rN/view?usp=sharing

Here’s the layout of parts on the hat, from the middle of my adventures in sewing earlier this week:

Lilly’s Status Report for 3/29/2025

Items done this week:

  1. Fully fixed the BLE communication from ESP32->RPi (ability to send the full float at once instead of separate bytes, which was a problem for some reason last week). Pi->server communication for sending over the angle data as received from the ESP32 is set up as well.
  2. Tested sensor fusion algorithms to improve the angle calculations using the acceleration, gyro, and magnetic field data). These readings are still a bit buggy so I still need to implement some filtering on the sensors readings + calculated outputs so the values being sent to the extension do not jump up and down so much. However, they’re still a lot better than last week and get a lot closer to the correct angle, especially when moving the sensor slowly.

Progress?

I had hoped to have much cleaner angle data by now so I would say I’m still behind on that front. I also did not have time to fix the Pi->ESP32 sending issues as I focused on the angle calculation algorithms more this week. However, this is not that much of a problem as the main thing that the Pi/server needs to communicate with the ESP32 is when to start calibration, but this can just be synced with the timing of the Pi’s connection to the ESP32 via Bluetooth (instead of sending some value to indicate the calibration start). Still, I find this messy and I would like to implement the Pi->ESP32 control in the coming weeks so the “user” (and us developers…) can recalibrate/restart sampling via a command in the browser extension. I also decided to not start assembling the hat since our real visor order hasn’t come in yet, and I didn’t want to waste time making something that wouldn’t be in our final product, even though the interim demo is on Monday.

For the interim demo, I would like to have a cleaner (and more accurate) stream of angle data available to send to the Pi/extension, and implement some way of zeroing out the angle/recalibrating mid-run (ideally controlled from the extension).

Deliverables for next week:

  1. Further improve angle processing code (probably need more low-pass filtering on output and input data)
  2. Start assembling the components on the hat when the visor comes in.
  3. Figure out the BLE (Pi->ESP32) issues so calibration can be more streamlined.

https://drive.google.com/file/d/1wddo5KdtA5FNAKvYoB1es1snJE4sNH3T/view?usp=sharing 

Video of the calculated angles when rotating the board from 0 -> 90 degrees in one direction (board offscreen). You can see that it’s still a bit off (only reaches -80 to -85 degrees at the “90 degree” point) and oscillates quite a bit before stabilizing when I stop moving. One good thing is that I’ve calibrated the gyroscopes more extensively so there’s a lot less drift (changing angle when the sensor isn’t even moving).

Lilly’s Status Report for 3/22/2025

Items done this week:

  1. Found/ordered a hat + mesh fabric for putting all head components on. Unsure if this will arrive in time for interim demos (my bad) so a rough prototype can just use a regular baseball cap + scrap fabric to attach parts to in little “pockets”.
  2. Worked on two-way BLE communication between RPi and ESP32. Sending data from ESP -> RPi works properly (need to format/process the data on the Pi’s end though), but ran into some troubles with sending control data from Pi -> ESP32 unfortunately so this will require some more time researching + debugging.

I am still a little behind as debugging BLE issues took up a lot of time, and will still require more time as of the time of this report. For the same reason I did not get a chance to finish implementing more advanced filtering strategies to process the angle data, which will be my first priority next week/tonight/tomorrow since getting good angle data is more important than having nice synchronization for the calibration (we can just “synchronize” manually by turning the ESP32 on/off when we are ready to receive data for now).

 

Deliverables for next week:

  1. Finish angle processing code.
  2. Start assembling hat/pockets using the spare ESP32/gyro/battery (likely need to solder differently to make the components less stabby for the user + make wiring more streamlined).
  3. Figure out the BLE (Pi->ESP32) issues.
  4. Start on Pi->server communication for sending over angle data. This shouldn’t take too long since it is the same thing as the footpad sensor communication.

 

Here’s a screenshot of the start of the handshaking for the calibration flow over bluetooth. We are able to send data to the Pi but the other direction is not done yet, as you can see from the stalling when we try to start the calibration process.

Lilly’s Status Report for 3/15/2025

Items done:

  1. Set up and tested BLE communication on the ESP32. This works! Also refined the user calibration flow based on receiving values from the BLE sender. Actual angle calculation is better than last week but still a bit rough (calibrated offsets are not fixing the problem of drifting angles completely – filtering not implemented yet…).
  2. Started setting up BLE scripts on the Pi as well. Testing it right now, should be more or less good to go by the end of the weekend. Currently the RPi is able to connect to the ESP32 as a client over bluetooth which is nice. At the very least, two-way communication on the RPi should be set up within a couple days even if the interface between the ESP32 and RPi is still buggy.

Progress on the angle calculation code is still a little behind, as more time was spent figuring out the BLE communication. But I believe this should be finished by ~Wednesday since we have Ethics on Monday. Progress on setting up the BLE communication is fine as of now, though not fully done yet. One problem is that I did not yet put in an order for the mesh fabric/visor, as we spent more time discussing the design of the neck system over the past week. This will actually be done by Monday so the parts can be ordered on Tuesday.

Deliverables for next week!

  1. Finalize the angle calculation code using a filtering algorithm
  2. Order a hat + mesh fabric for the neck system, finalize design for this as well. 
  3. Finish setting up BLE communication between ESP32/RPi (set up packet sending/receiving protocol now that connections are established).
  4. If the above are done, start integrating with Kaitlyn’s code on the RPi as well so we can run everything together.

Sample calibration flow, driven by a command over BT: Receive calibration command –> average the gyro/accel data over some number of samples to calibrate the “no movement” level –> start calculating + sending angle data

 

Connection w/RPi: right now this is just connecting, not sending anything

Team Status Report for 3/15/2025

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

  1. Getting the different systems using the Pi to run at the same time and still meet our latency requirements. We are still working on unit testing each part but we will be putting the code together (hopefully) ~next week. If the system becomes really slow with the RPi bluetooth sender/receiver running, we’ll have to find places to simplify our code and the data being sent to the server at a time. Currently the latencies for sending just the foot sensor readings to the server and the extension reading data from the server are pretty small, so we are not too worried. Power draw may also be a concern when we have more computation being done,  since our Pi already gets wickedly hot before we even run anything. In terms of safety we can manage this risk by keeping the Pi very far from the user when we put the mat together. We thought about having some sort of casing to protect the user, but realize this would also worsen the heating problem.
  2. Managing the bluetooth connection between ESP32 and RPi. Setting this up is taking a little longer than expected (at least as of the time of writing this team report – the work continues today…) due to some unexpected incompatibilities with some of the packages we tried to use. However, we’ve already successfully tested two-way communication from the ESP32 and the nRF connect app (just downloaded on a phone), and we’ve set up a BLE scanner on the RPi with compatible libraries to test today. If we completely fail at setting this up, we could try connecting to the server via Wifi directly to send/receive data. However, this is probably way more work than using bluetooth (and would likely destroy our power requirements) so this is more of an absolute-worst case plan. We also expect the BT to be figured out soon, as there is a decent amount of documentation on this.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

Instead of using a breadboard to do the wiring of the foot sensor mat, we’re using a protoboard with a 3/5V regulator + voltage divider circuits + pin headers soldered on. This was needed to make connections easier to reconfigure and keep our wiring more organized/scalable (even just connecting 2 foot pads using the breadboard was already pretty burdensome). No costs were incurred since we already had the protoboards from a previous project.

Provide an updated schedule if changes have occurred.

As of right now, no schedule changes are needed. Integration is underway to set up communication between all our components, though we are still also working on getting the data sent from each part to be what we want.

Lilly’s Status Report for 3/8/2025

Items done:

  1. Wrote/tested initial angle calculation + user calibration code for the ESP32. Changes in angles reported are broadly correct but a lot of refinement is needed to make this usable.
  2. Finished the design report, which ate up most of the week before break.

 

Unfortunately my progress is a bit behind schedule, as I had aimed to have written and tested a basic BLE script for sending data between the ESP32 and RPi by now, and have a more stable version of the angle calculation code as well. I think the angle calculation code shouldn’t take too long since I know what I want to implement to fix the drifting values, so I will likely have enough time this week to figure out the RPi communications after coming back from break and getting a midterm over with. I had also wanted to put in an order for the non-electronic hat parts before the break, but this will be done on Monday now. We may also use a random hat we have, and fabric from Techspark to prototype for now if these parts do not arrive soon.

 

Deliverables for next week!

  1. Add complementary filtering to the angle calculation algorithm, and try offsetting drift values in the code. Also see how much the data improves from averaging over two sensors to decide if this is worth doing.
  2. Write BLE script for the communication to the RPi and set up calibration flow in the code (need to work with Cora for UI parts).
  3. Order a hat and mesh fabric for the wearable parts.

Team Status Report for 3/8/2025

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk is currently the integration between the sensors and the browser extension. Specifically, sending signals to and from the extension to the sensors, and this updating the workings of both accordingly. These risks are being managed by working on this integration early – the local hosted server is already set up and the requests to / from the sensors to the server have already been implemented. We plan on working this week on integration, and dealing with any challenges as they come up, specifically being able to pivot from the Node.js which we are currently using to flask if needed to ensure the integration works.

 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

A change was made to how the sensors are interacting with the RPi. Due to a discrepancy found between the datasheet and the ADC parts, we found that we could not use 4 adcs are originally intended. Instead, we are using a 16:1 MUX and a singular adc, and selecting which input from the mux to read, then forwarding that to the ADC and then the Pi. This did not incur any costs, as one of the team members was able to scavenge this mux and therefore we did not need to order anything.

 

Provide an updated schedule if changes have occurred.

As of right now, no schedule changes are needed. 

 

Part A was written by Kaitlyn:

With consideration of global factors, StrainLess can impact people outside of Pittsburgh. Since the product can be easily reproduced, it can be used around the globe, not only at CMU. Anyone with access to Chrome is able to download our extension, and making / potentially buying the pressure mat which comes along with it is feasible. Even with people who are not technologically savvy, after having installed the browser extension they will have a very friendly user interface with charts and graphics which are easily understandable. As well, people who are not in an academic environment can still use our product, as it is meant for anyone who spends time at a desk – this could be office workers, gamers, or people who work from home.

 

Part B was written by Cora :

StrainLess will support the 9-5 work culture that exists in the United States by improving the quality of life of those who work at a desk. Being hard-working and self-sufficient are core beliefs held by people from the United States that many people feel define who they are and StrainLess will be conducive to this way of thinking by reducing the pain of these people that they likely feel they just need to deal with as part of their lives. StrainLess’s UI will be in English, but with intuitive controls and notifications so even if people do not speak English if they have a good idea of what the product does, they will still be able to use it. Regarding laws, StrainLess does not break any laws and in fact could improve workplace safety if implemented in an office setting.

 

Part C was written by Lilly: 

The main environmental goal that StrainLess supports is waste reduction by not having disposable parts in our final user product. While we also are designing our wearable parts to use low power, this is more for safety and functionality than environmental considerations since the magnitudes/scales for microcontroller power consumptions overall are very small. But, we have avoided the use of any disposable components in our product by using rechargeable batteries instead of single-use ones, making sure to include a charging port on the microcontroller board so the user does not have to acquire more materials to charge. All other components are powered by a wall/outlet supply or are just run on the user’s computer’s battery so they would not require any disposable power supplies either. Although LiPo battery recycling exists, it’s not available everywhere, and reduction of resource consumption in the first place tends to be more sustainable than recycling since the recycling process in itself is imperfect and takes up a lot of energy anyway. This reduces our product’s (negative) environmental impact by not requiring the user to have to dispose of batteries, which can be very toxic to the environment if done haphazardly. 

Lilly’s Status Report for 2/22/2025

Items completed this week:

  1. Tested the ESP32 microcontroller to make sure it worked alright (could be flashed with an example program, didn’t overheat). Researched different development environments (PlatformIO, Arduino) but ultimately decided to stick with the Arduino IDE for simplicity and ease of connecting to our specific microcontroller. I initially had concerns about the compiled code size using the Arduino IDE vs. other platforms but there should be more than enough memory on the board to handle this!
  2. Presented our design review to the class.
  3. Soldered pin headers to the ESP32 and the LSM9DS1 sensors so I can start breadboard prototyping and setting up I2C communications between them.
  4. Researched how to set up the BLE connection from the ESP32.

The I2C setup will hopefully be done tomorrow (on Sunday)! Along with connectivity checks for my soldering… Otherwise I’m pretty on-schedule, especially since I have all the necessary parts now.

 

Deliverables for next week:

  1. Finish up the design report.
  2. Test sending data over BLE between RPi and ESP32.
  3. Code up initial filtering/data processing algorithms for the gyro sensor data.
  4. Design mounting system for hat.