Carnegie Mellon ECE Capstone, Spring 2025: Kaya Akinci, Cynthia Meah, Maya Voelkl
Accomplishments this week:
This week, we worked on all of our testing, and the only thing we have left to test is latency. We also implemented a feature for detecting downward stairs.
Reflection on schedule:
On schedule!
Plans for next week:
Over the next week, we will be working on the poster, final video, and any last-minute adjustments our device may need.
Accomplishments this week:
This week, I improved the assembly of the cane by securing everything in a less temporary way. This included some woodworking for our camera holder and some additional aspects to hide our wires and such. I also improved the haptic feedback responses to be much stronger. The second half of the week was spent testing our prototype and putting together parts of our final presentation.
Reflection on schedule:
On schedule!
Plans for next week:
Over the next week, we will be completing the rest of our testing and documentation and getting some materials together for our demonstration. This includes stairs, figuring out lighting, and putting together our poster.
New tools:
On the hardware side, I became familiar with Adafruit’s QT Py RP2040 microcontroller, and the DRV2605L haptic driver, which required us to learn how to communicate between these two devices and with the jetson. Additionally, I learned how to use FSRs to trigger our threshold for the CV.
To gain this knowledge, I relied heavily on informal learning strategies. I used GitHub example code, Adafruit and NVIDIA forums, and documentation pages to understand how each component worked and how to debug integration issues. We also used trial-and-error testing and peer troubleshooting within our team to identify bugs and refine our software logic, especially when integrating the camera, haptic feedback and FSR with our Jetson.
What are the most significant risks that could jeopardize the success of the
project? How are these risks being managed? What contingency plans are ready?
The risks that could jeopardize the success of the project is the battery not working for the Jetson Orin Nano and it possibly frying our Jetson. We performed extensive research on this to make sure it won’t fry it but for the small chance that it will, we backed up all of our code on github and have all of the individual components. working seperately
• Were any changes made to the existing design of the system (requirements,
block diagram, system spec, etc)? Why was this change necessary, what costs
does the change incur, and how will these costs be mitigated going forward?
• Provide an updated schedule if changes have occurred.
• This is also the place to put some photos of your progress or to brag about a
component you got working.
There have been no chances to the existing design. There have been no changes to the schedule.
We have finished composing our cane!
Now that you have some portions of your project built, and entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?
Validation
We plan on testing all of the various main features individually and then together. This means testing the object detection on 50 various objects, testing the wall detection on 50 various walls, testing the FSR on 25 different surfaces. For the various object and wall tests, the haptic feedback should detect on 48 out of the 50 tests respectively (~95%). For the 25 different surfaces, we want the FSR’s to detect the floor on 24 out of 25 of them (~95%).
Additionally, we want the system to only have false positives <= 5% of the time.
Accomplishments this week:
This week, I wrote the code for the FSR, and soldered all of the connections together which means our entire cane composition is mostly complete. This included everything for the haptics, and both FSRs and their connections with the QT Py. My code currently prints out when each pressure pad is pressed or not pressed, and we are currently working to integrate these responses with the Computer Vision.
The left image is the feedback we get with pressing and releasing the FSRs. The right picture is how all of our wiring is set up with the cane.
Reflection on schedule:
We had a lot of important progress this week, and everything other than our power supply has finally come together. According to our schedule, our entire cane should be complete by Wednesday, and we should begin completing usability testing, so I think we are on track with that as long as the barrel jack converter we ordered is the fix to our power issues.
Verification:
FSRs:
HAPTICS
OVERALL:
Accomplishments this week:
This week, I integrated the haptics with the Jetson, cut the wood and materials for our cane, and helped Cynthia work out specific cases for when objects should be detected and to trigger the proper haptic signal.
Reflection on schedule:
We had a lot of progress as a group this week and we were able to get many big steps put together. We are in a great spot for the interim demo and are in a great spot in terms of our schedule.
Plans for next week:
Over the next week, I will be working on testing each of the elements for power and current to make sure they are all going to be safe before plugging them in to the portable charger. I will also work to set up the pressure pads if time allows this week, but it is less important than the power consumption.
Accomplishments this week:
This week, I setup the haptics and created a few sample patterns that we will be using with our Jetson. The haptic patterns for each obstacle type is demonstrated here.
Reflection on schedule:
We were a bit behind schedule at the beginning of the week because we had some problems with the Jetson and L515 compatibility, but we put in a lot of hours this week to recover from this. Personally, that included me helping Kaya with the Jetson and L515 connections, and I also setup the haptics and created case statements for when each haptic pattern is set off.
Plans for next week:
Over the next week, I will be working to run the haptics over the Jetson and hopefully begin to connect the haptic responses through the CV code, and I will begin testing the overall power consumption to make sure it is under 30W and 5V.
Accomplishments this week:
This week we linked the Jetson and the L515 using the SDK viewer, which is pictured below, and we are continuing to set up the L515 on the Jetson with our current code instead of the viewer code.
Reflection on schedule:
We are on schedule for the most part, but we have a heavy workday on Sunday to finish our goals for this week, which includes starting to integrate the haptics with the Jetson.
Plans for next week:
Over the next week, we will be working on the haptic logic and making sure it integrates with the Jetson.
Accomplishments:
This week, we found out that our Jetson would consume more power than we had initially planned for, so I spent a lot of time researching new power options that met our 5V, 6A power requirements of a portable charger. Kaya and I also worked to set up the Jetson Nano. Lastly, I did a lot of the final documentation and diagrams of our Design Review.
Progress:
We are on schedule now that we have finished the Design Report, our Jetson initialization, and L515 camera set up.
Future deliverables:
Our adafruit order was delivered, so I will be able to start working on the haptic vibration motor and starting to create the logic for different feedback patterns. Since Kaya and Cynthia will be working together on the software behind the computer vision, I plan to focus on more of the Jetson and haptics.
Risks:
Our most prominent risks right now are setting up the RealSense SDK, and troubleshooting any connection or compatibility issues that we may have with the Jetson.
Changes:
We initially had Maya and Kaya working together for most of the hardware components, but we are going to have Kaya and Cynthia work on more of the Computer Vision together as we changed our implementation to use a more challenging algorithm, and Maya will continue working on the hardware components. We are in a better spot with the Jetson than the RealSense and we are realizing that object detection is likely going to take longer than we planned, so we are giving that aspect more attention because many other aspects are reliant on the object detection being completed.
Accomplishments:
This week, we completed our design report, so we only have a few small details and tables to add and change before it is due on 2/28. My specific role in this was use-case requirements, part of design-case requirements, and a large part of the implementation plan. I also worked with Kaya to flash our Jetson for the first time.
Progress:
We are ahead of schedule in terms of our design report, and we are technically still on schedule for our technical advances, but it feels like we need to make a lot of progress with the jetson this week to stay on schedule.
Future deliverables:
In this upcoming week, we will be finishing our design presentation and finalizing all of our tables and diagrams that go with that. We will also be working a lot on the Jetson and hopefully beginning to look at the FSR and haptic motors if they come in this week. This week needs to be much more tech-focused than documentation focused so that we are in a good place after spring break.