Posts

Posted on

Final Team Update for 12/13/2024

General Update

Drumroll please……….We won!

  1. (30hr Frontend Stretch): Alex and Jubi worked on ironing out bugs in the bluetooth communication, dynamic button generation, chart displays/analysis, and threshold updates.
  2. (30hr Hardware Stretch): Josh worked on designing and assembling a PCB (stretch goal), designing a CAD model to house the device hardware (stretch goal), and creating a second glove (stretch goal). Another stretch goals Josh accomplished was integrating dynamic haptic vibration that increases as the force readings encroach dangerous levels.
  3. (18hr Video Scripting, Recording, & Editing): This week, Alex, Jubi, and Josh spent time writing a script, recording, and editing a final video that demonstrates our product and design process.
  4. (20hr End-to-end Testing): This week, Alex, Jubi, and Josh continued testing the device at the Iron City Bouldering gym,  where we conducted end-to-end tests using the app to begin, end, and analyze workouts using the CLIMB device.
Risks and risk mitigation

None!

overall design changes

Towards the end, we worked towards supporting our ultimate stretch goal of DUAL WIELDING, however, this would require significant additions and possible revamps of the front-end to account for visualizing data from two hands.

schedule

The schedule remains the same as we are done!

GANTT Chart

Guidance

This project was so fun! Alex, Jubi, and Josh had a great experience desiging CLIMB and being able to demonstrate the culmination of our ECE undergraduate experience here at Carnegie Mellon University. Throughout this project, we utilized the cornerstones of ECE: circuit design, signal systems, hardware, and software systems. In doing so, we developed a durable, unobtrusive system of assistive features for rehab/training of climbers: real-time pulley monitoring, pulley-risk indication via haptics, wireless, battery-powered use, and workout tracking/analysis via a robust, intuitive user interface (the CLIMB app). We are very proud of this work, and will take this experience with us past graduation. Thank you to everyone who supported us throughout this process, especially the course faculty, TAs, and fellow classmates who provided invaluable feedback at every step!

Posted on

Jubahed’s Status Report for 12/7/24

  1. Mobile App Development (10h):
    1. Charting (4h): Integrated charting graphics into app, on top of the queries that Alex wrote to pull data from the database. Added some new queries to ensure safety in all edge cases (app used to crash when a query returns something incomprehensible or a query is wrong!)
    2. Mandatory Lab Meetings (4h):  Provided feedback to other teams presenting their final demo slideshows.
    3. Final deliverables (2h): Started working on final deliverables.
Progress
  1. Progress is on schedule.
Next Week Tasks & Goals
  1. Finish up final deliverables and prep for final demo!
  2. End-to-end last minute testing in HH room where demos will take place
Posted on

Team Update for 12/7/2024

General Update

This week Josh, Alex, and Jubi worked on the final paper, poster, and video, continued verification & validation tests on the hardware & software solutions, and finalized work on the device integration / continued stretch goals. Josh worked on designing and ordered a PCB to house the hardware for the device.  He also began working on the CAD for the hardware capsule. Alex worked on data visualization within the mobile app as well as adding app functionality for the second glove (BT/calibration modifications, frontend modifications). Jubi worked on tweaking the recent queries add for edge case safety, as well as integrating in the visual displays for charting / pulling data from the database.

Risks and risk mitigation

None

overall design changes

None

schedule

The schedule remains the same as we are on track.

GANTT Chart

Guidance

List all unit tests and overall system test carried out for experimentation of the system. List any findings and design changes made from your analysis of test results and other data obtained from the experimentation.

We performed static weight unit tests via arm hangs on the sensors, from which we realized that the best placement of the sensors are on the A2 and A4 pulleys, the tendons between the first two knuckles stemming your middle and ring finger. This is because we realized that most of the weight is distributed on these pulleys, and although pulley injuries can still occur in the A2 and A4 due to pressure placed on the A5 pulley, during rehabilitation, it is important that minimal force is placed on these injured A2/4 pulleys.
Posted on

Joshua Ramos’ Status Report for 12/7/24

Personal Accomplishments
  1. Final presentations (4hr): This week I spent time writing peer reviews for final presentations.
  2. PCB design (stretch-goal) (4hr): This week I spent time designing a PCB for our device using Fusion 360. This PCB houses an Olimex ESP-32 breakoutboard, exposing 4 adc and pwm gpio ports for our device use. It also contains the sensor amplifier circuit.
  3. More Testing (4hr): This week I spent more time testing the device. I performed more ergonomic testing via survey, durability testing via static and dynamic hangs/climbs, and performance tests with the battery.
  4. Final paper/poster writing (4hr): This week I spent time working on the final poster and paper, filling in an outline and coming up with a rough draft.
Progress
  1. Everything is on track! Working on stretch goals (PCB, CAD, Second Glove, etc.)
Next Week tasks & goals
  1. Work on CAD for device capsule (stretch-goal)
  2. Continue testing
  3. Continue writing final paper and poster.
Posted on

Alex Nguyen’s Status Report for 12/07/24

Personal Accomplishments
  1. Mobile App Development (8h):
    1. Data Visualization: Spent time adding data visualization capabilities to the mobile app. This consisted mostly of modifying and adding queries and database functions to grab relevant session data from the database to display.
    2. Double Glove: Mainly worked on our first stretch goal, adding a second glove. For the app, this mainly consisted of adding/modifying the necessary frontend elements and interactions. The bluetooth sequence may need to be modified further to handle one or two gloves without failure occurring when it only detects one glove.
  2. Mandatory Lab Meetings (4h):  Gave our team’s final presentation, in addition to providing feedback on other teams’ final presentations.
Progress
  1. Progress is on schedule.
Next Week Tasks & Goals
  1. Final Project Documents: Finish final video, final report, and prepare final demo.
  2. Complete Integration Testing: integrate every piece of the project, including the second glove, and complete all final unit tests and integration tests for the report and poster.
Posted on

Jubahed Qayum’s Status Report for 11/30/24

PERSONAL ACCOMPLISHMENTS

  1. Lab meetings (6h): My team and I presented our interim demos. We got great feedback and thought it went well!
  2. App backend + frontend merging (6hr): Last week, Alex’s frontend elements were isolated and did not utilize bluetooth connectivity or commit any data to device storage. Therefore, I worked with Alex to merge together my backend APIs (Bluetooth + Database commitment) with his frontend elements. This made sure that the user experience for the device was significantly more ergonomic and seamless.
  3. ESP32 programming (3hr): I showed Alex my ESP32 toolchain setup and explained my FSM backend process on the ESP such that Alex would be able to start working on the calibration sequence as easily as possible.
  4. Device Verification & Validation (6hr):  Alex and I conducted a variety of software unit + integration tests to ensure the robustness of our app. This included bluetooth distance testing (mocking the physical glove with dummy data and ensuring the app stayed alive when the device moved 15m+ away), making sure that buttons could not be spammed and that the FSM of the app could not enter any unknown state (if a user presses a button twice within 5-20 ms, before an animation sequence occurs, for example, then the app should not start any code sequence twice). Since the migration to the frontend elements removed aspects of debugging I had prior, like displaying database elements properly, we learned how to debug the app using console logging via USB and displaying the database through Android Studio’s logcat and Database Inspector tools. These were used to make sure data was properly being committed to the database.
  5. Final Presentation prepare (6hr): I helped my team put together the final presentation slides, specifically information regarding the HW/SW interface, Bluetooth, and data storage.
PROGRESS
  1. Continuing testing into a more-than-MVP state
  2. Everything is on schedule!
NEXT WEEK TASKS & GOALS
  1. Alex and I need to sync up again to make sure that the app frontends can properly **retrieve and display** data from the database with specific queries. I’ll need to work closely with him to make sure my backend APIs are clear and further more robust for his use.
  2. Integrate second glove into backend APIs. This will include a lot of redundancy but will still take a sizable chunk of time.
  3. Continue testing, get more involved with end-to-end testing
  4. Work on final report
GUIDANCE

I think I learned a lot of new skills working on this project, but more specifically I learned about both app development and the process necessary to build an app effectively for Android, as well as the general ideation + design process for a project of this caliber, especially when it comes to deciding what components to use in the first place.

With respect to Android development, prior to this project I had actually never created an Android app before. Therefore, it was definitely foreign territory and a significant amount of time was spent on trying to make sure all elements of the app were working, plus just figuring out how to navigate Android Studio in general took a long time, even before assembly of our physical artifacts began. I learned the Kotlin programming languages, how Android breaks app pages into ‘Activities’ and how those activities correspond to frontend and backend elements, and how to utilize device permissions to do awesome things (like geolocation or Bluetooth access).

With respect to strategies to tackle this sort of project again, I really think I learned how to effectively pick and place components. By this, I mean that at the start of the project, my team really started from a blank canvas, where we had to decide entirely on our own what microcontroller to use, what sensors to use, whether we should develop for Android or iOS, etc. Before Capstone, these sorts of problems were already solved for me, where someone else had done that work already, but now being in the shoes of that person, I can see why this is a difficult and daunting task. We initially stumbled around, trying out different hardware platforms like Arduino, STM32, etc., until we landed on ESP32 (and even then, the first devboard we decided to trial on didn’t support power inputs the way we wanted). This process of iteratively building our design and our components over time + over trial and error taught me that

  1. Your design doesn’t need to be perfect off the jump, but in fact that this is a growing item and can change as the parameters, scope, and level of understanding around what we want to project to obtain or achieve changes
  2. Research really is necessary before any type of assembly or work. I know our team was itching to put together physical + software artifacts ASAP, but I distinctly remember that during our design report work sessions, we genuinely had points where we realized that we had NO CLUE how to pursue a subaspect of the project and had to spend time understanding it. Sure, the funnest part of the project was the assembly, which we only really did toward the end of our project, but the prior research and acquisition  of knowledge from our past reports really was necessary for us.
Posted on

Alex Nguyen’s Status Report for 11/30/24

Personal Accomplishments
  1. Programming ESP32 (5h) – I programmed the calibration routine for the ESP32, such that upon session startup (and immediately following a confirmed bluetooth connection) the calibration routine will launch and take the average of the next 10 seconds of force readings for each sensor, storing that value as the normalization value for the rest of the readings taken during the session. This can be seen below, as the normalization values for each sensor are displayed upon completion of the CR at 10 seconds, and then the device proceeds to take readings for normal data collection. 
  2. App & Bluetooth Integration (12h) – I worked with Jubi to integrate the front end with bluetooth and database. Once Jubi established a working bluetooth connection between the device and the app, I realized that our current code was overly reliant on disconnected Kotlin coroutines, and was able to simplify them into two larger Kotlin coroutines and suspending sequential coroutines until prior ones had finished which prevented many of the bugs that were occurring when two coroutines overlapped or attempted to access data at incorrect timings.  When I completed the calibration routine on the ESP32, I synchronized the bluetooth connecting screen and calibration timer screen to match the events occurring on the ESP32 (with the bluetooth connecting screen staying until either it was connected or timed out and the calibration timer matching the timer of the ESP32).  This can be seen below where the light to indicate bluetooth connection is blinking before the session starts, then turns solid when a connection is established (matching the changing of screens in the app). 
  3. Android App Development (frontend) (10h) –  Since the last update, I implemented the rest of the frontend that was remaining. This included the threshold settings page, questionnaire, and session history selection page. I also attempted to query Jubi’s database for the max sensor reading values recorded for each pulley during a session, however it is currently inconsistent and returns either the same values or random values, which is something I will work on further. Assuming this is not an issue with the database, I believe this is most likely an issue with data sharing between the MainActivity and the HomePage, potentially caused by uninitialized key-value pairs in sharedPrefs.
  4. Integration Testing (5h) – Went to the climbing gym with Josh and conducted a series of tests on different climbing apparatuses (hangboard, pullup bar, campus wall, bouldering wall). We were able to verify several key factors, including bluetooth connection between the app and device, as well as test the reliability and consistency of the sensor when doing various climbing movements. Additionally we were able to test the usability of the device, both by climbing on routes with the glove on as well as surveying several climbers who were also at the gym.
  5. Final Presentation (8h) – I am presenting this week, so I worked on the presentation slides and prepared to present during this week’s lab meetings.
  6. Mandatory Lab Meetings (6h) – Presented intermediate demos to professors and TAs and received valuable feedback, especially regarding the data visualization and analytics components to our mobile app.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Before this project, I had never used Android Studio or Kotlin before, and I was pretty unfamiliar with app development model. As a result, I had to spend a lot of time at the beginning of the semester experimenting while building the app, even before our team had a concrete plan as to what features we wanted to have as part of the mobile app. I found tutorials online very helpful for learning the building blocks of app development, and learned a lot through tinkering with different interactions between frontend and backend elements in dummy applications that I created just for the purpose of testing out potential interactions I could want in the final app.  This learning process was a lot of trial and error, so git version control was incredibly important as there were several instances where I had to rebase after creating an error whose origin I was unable to identify.

Additionally, I learned a lot about the design process for an embedded system mobile app. Prior to this project I had never had to consider both factors when designing a product, so there were many new challenges that I had to consider (along with my teammates). While we were writing our design report, we realized how much of our design we had not considered regarding the interactions between each of our individual parts. Sitting down and talking through the overall interactions between the hardware, frontend software, and backend software, was immensely helpful, and while it took a long time, it proved to be invaluable to the success of our process later on when we began integrating. This skill was developed further throughout the semester, as we made sure to revisit (and modify if necessary) our overall integration design plans each time we made major changes to any one individual part.

Finally, designing thorough unit and especially integration testing was a skill that I found very important to learn throughout the project. While I have conducted unit testing for software projects before, the addition of a connected embedded system communicating via Bluetooth added an extra dimension of testing needed between all three systems (embedded device, BT communication, backend database, frontend UI/UX). This skill was learned on the go, as we each performed our own tests on our individual parts but had to figure out how and what to test during integration. Figuring out the many aspects that needed to be verified and validated was a process that often happened accidentally, as one of us discovered a factor that had not yet been considered (things like finger placement affecting sensor readings, BT connection failure during a preset sequence, race conditions occurring when a coroutine terminated early). Finding new factors that could adversely affect the optimal performance of our product taught me both how important it was to design tests for as many of these potential issues as possible as well as how to design appropriate tests to ensure our goals were met.

Progress
  1. Progress is on schedule.
Next Week Tasks & Goals
  1. Data Visualization and Analysis: Work with Jubi to retrieve data from database and display based on session number.
  2. Dual Glove Additions: Make any additions necessary for Bluetooth and Mobile app with the introduction of a second glove (likely frontend additions, some communication changes)
  3. Final Report: work on final report with teammates.
  4. Testing: continue testing as the final product comes closer to completion, especially with regards to new additions or aspects that were modified. Additional reliability tests and real-world application tests (in climbing gym).
Posted on

Team Update for 11/30/2024

General Update

This week Josh, Alex, and Jubi worked on the final presentation, performed verification & validation tests on the hardware & software solutions, and continued work on the device integration / began some stretch goals. Josh worked on tuning the amplification gain/range to provide more robust and consistence sensor readings. He also created a second glove using the new Olimex ESP-32 and tested/confirmed battery-powered functionality. He also worked with Alex to perform some verification and validation tests at the Iron City Bouldering Gym, and he began working on a PCB design and device capsule CAD. Alex primarily worked on programming the calibration routine on the ESP32, as well as rewriting front end code for the startup sequence (Calibration Routine, Bluetooth Connection) to avoid coroutine-related errors and sync the states on the ESP32 to match the timing of the mobile App frontend. Alex also conducted testing focusing on the interactions between the app and the wearable device during a full session from beginning to end. Jubahed assisted Alex with software end-to-end testing, as well as explaining his APIs to make sure that Alex could properly build the calibration system upon them. He also continued work on the back-end database system, as well as integrating bluetooth connectivity between the device and app. This mostly included integrating the prior frontend elements with the backend APIs, and ensuring that they worked together as intended as smoothly as possible.

Risks and risk mitigation

None

overall design changes

We are now using the Olimex ESP-32 LiPo DevKit board with an on-board power-jack because it allows us to provide battery-powered functionality to the CLIMB device without needing to design an LDO circuit on the PCB, though this solution is a bit more expensive than the previous ESP-32 we were using from AdaFruit.

schedule

The schedule remains the same as we are on track.

GANTT Chart

 

Posted on

Joshua Ramos’ Status Report for 11/30/24

Personal Accomplishments
  1. Lab meetings (6h): During our schedule lab meeting hours, we did our interm demo and recieved great feedback. I also continued work on the hardware solution, which is described below.
  2. Second glove implementation (4hr): Worked on creating a second glove. This consisted of copying the first glove in implementation, but instead using our new Olimex ESP32 with an on-board power jack so we can test a battery-powered implementation. I spent a deal of time soldering up 4 more sensors with wiring such that they can be attached to the second glove. The battery powered ESP works!
  3. Built pull-up bar for final demo (1hr):  I spent time building the pull-up bar we ordered for our final demo.
  4. Sensor tuning (3hr):  I spent time deciding what gain/voltage range we wanted to use for our sensor amplifier, after testing whether a 0.5-3.3 vs. 0.25-3.3 vs 0.15-3.3 voltage range, using a gain of (837k/[3M->100k]), I found that the larger forces were better measured, since the the gain was more linear, using the 0.15-3.3 range.
  5. Device Verification & Validation (5hr):  I spent time working with Alex to run verification & validation test at the Iron City Bouldering gym where we performed durability, safety, ergonomic, and BT speed tests using the open climbing environment with various climbing levels and hangbars for more static weight tests.
  6. Final Presentation prepare (8hr): I spent time working on the final presentation, specifically updating the solution body diagram, the hardware solution, the testing requirements, the hardware/software test results,  and design tradeoffs.
Progress
  1. Continuing work on verification and validation tests
  2. Everything is on track! Working on stretch goals (PCB, CAD, Second Glove, etc.)
Next Week tasks & goals
  1. Working on PCB and CAD for device capsule
Guidance

I learned how to find an appropriate microcontroller for a given application and how to use a new one: ESP-32. Previously I had worked with an STM32 and Teensy 4.1, but those were in projects where the MCU was chosen for me. So for this project, I had to do some research into viable MCU’s, where I learned how to perform proper tradeoff analyses for deciding which hardware to use for my application. I spent a great deal of time deciding which factors I should use to determine viability, and in doing so, found myself learning how to scan through reviews and documentation to perform comparisons of each device. The same occured in my process for finding which sensors we should use for our CLIMB device. I even had to call sensor distributors to acquire more information about the sensors I was considering for my application. This process forced and taught me to become more comfortable in information gathering and tradeoff analyses for the application design process. In regard to learning how to work with the ESP-32, I learned how to work with the Expressif toolchain to flash and debug my CLIMB code. I also worked with the Arduino libraries to utilize the on-board ADCs and PWM modules, which required me to look into the MCU’s peripheral functionalities, and decide which ports I would use for the various tasks my application required.

Posted on

Team Update for 11/16/2024

General Update

This week, Josh, Alex, and Jubi spent most of their time working on preparing the product for demo. This consisted of Josh modularizing the prototype with plug-and-play sensor functionality, soldering the sensor wiring and heat-gunning insulation covers instead of using jumpers for a clean look, running 4 sensors at once that connect to dc motors that vibrate upon threshold excession. Alex worked on the CLIMB user experience by designing the UI in Figma, and programming it in Android Studio. Jubi worked on implementing bluetooth communication between the ESP32 and the Android smartphone, achieving communication from the board to the phone and implementing a state machine that transitions the phone between start and stop modes. He also worked on implementing the database that will store the CLIMB metrics on the local device. Next steps are to achieve communcation from the phone to ESP, design a PCB to house the ESP and amplifier circuit, and data visualization and analyses to display on the UI. Also we will begin verification and validation.

Risks and risk mitigation

None

overall design changes

Using PWM to control the haptic motors which will have 3 vibration levels (1V, 2V, 3V).

schedule

The schedule remains the same as we are on track.

GANTT Chart

Guidance Question

In all, we are to perform ergonomic validation tests, durability verification and validation tests, safety verification tests, social/feedback verification tests, and finally end-to-end varification tests.

So far, we have performed basic hardware utilization tests such as working out with the glove on while monitoring the force readings, making sure that the readings are consistent all-throughout. The results of this test have been favorable, resulting in consistent readings.

(HW) Testing Durability

To verify the system’s durability requirements, the device will undergo 50 uses, each for 5 minutes with a constant 10kg applied at a 90 degree angle. This will be repeated for 50kg as well. After each, the sensitivity loss of the device will be verified to be within 2.5% and the wearable fabric will be examined for any tears or disfigurations. To validate durability, the device will undergo dynamic weight applied by a user at various angles for 1-hour, 10 times. After each, the sensitivity loss of the device will be validated to be within 2.5% and the wearable fabric will be examined for any tears or disfigurations. This meets the system’s durability use-case requirements by tackling even the most extreme cases (54kg/sensor) and verifying that the system remains robust.

(HW) Testing Ergonomics

To validate the system’s ergonomic requirements, a survey will be conducted on 50 randomly selected participants. Each participant will provide the shape characteristics of their fingers. With this information, the system sensor size requirements will be tested to only impose up to 30% surface area coverage for at least 90% of participants’ fingers. This meets the system’s ergonomic use-case requirements by providing a comfortable and unobtrusive wearable device that covers no more than 30% of a user’s finger.

(HW) Testing Safety
To verify that the system initializes properly, the system will be booted up and calibrated 50 times. The system shall successfully calibrate 95% of the time.
To verify sensor precision, incremental weights will be placed atop the sensor (1kg, 2kg, 3kg, etc.) and a voltmeter will be used to monitor the voltage changes per kg. The system shall reflect a 5mV/kg voltage change up to 54 kg. To verify that an alarm signal is triggered within 100ms when the force reading is within 20% of the safety threshold, an instant force exceeding the threshold will be applied after initialization, and an external timer will record the delay. The system shall fire its alarm within 100ms at least 98% of the time. To verify the second haptic’s functionality, the first haptic will be disconnected. Immediate force will be applied until the threshold is reached. The system shall fire its second alarm within 200ms of threshold excession with a 98% success rate. This meets the system’s safety use-case requirements by demonstrating that the system will actively work to prevent pulley injuries by remaining with a maximum error of ±2.5 and firing alarms within 200ms.

(SW) Testing Social / Feedback

To verify the feedback requirements of the system, tests will be performed using mocked force readings to verify the minimum data transfer. To do this, a mocked buffer input of 96 Kb will be generated and communicated to the mobile device. This test will be timed, recorded, and repeated 50 times to verify that the shipment completes in 10s with a 95% success rate.
To verify the user-experience related requirements, 96Kb of data will be mocked on the mobile device and passed to the database handler to ensure that information is stored properly and can be pulled to display on the application charts. This test will be repeated 50 times to verify that the flow of data on the mobile device can reach the user’s screen with a 95% success rate. These tests meet the use-case requirements by verifying that the system is providing users with a seamless experience in accessing analytical feedback that is suggestive towards improvement and can be used to compare progress with others.

(HW/SW) Testing End-to-end

To verify end-to-end integration, the system will undergo 100 simulations where mocked force readings will be generated and communicated to the mobile device, testing to determine if the data is successfully displayed analytically 95% of the time. To validate the system, a test using a hang-board with force-sensors mounted will be used to perform dynamic tests. The force on the hand-sensors will be cross-referenced with the force-sensors on the board. This test will be conducted 20 times for 10 minutes each. The system will be tested for the average error to be below 2.5%