Posted on

Cynthia’s Status Report 4/12

Accomplishments this week:
This week I spent most of my time debugging the fine-tuning code to make our system faster, which ended up not performing as expected, so further debugging will need to be done (but the current model is still working well, just slightly laggy). I also worked with Kaya to integrate wall detection with our code and get the correct response sent to the user.

Reflection on schedule:
We are on schedule, but because of laggy-ness our project will likely have a lower accuracy than our design requirement.

Plans for next week:
Testing and verification, further debugging, and starting our final report if we have time.

Verification:
I will focus on hazard and stair detection testing.
I will test the model (after removing the display of the frames which has been making the program slower) by analyzing the distance/location accuracy of objects detected, whether hazards vs non-hazards consistency get identified or not identified as expected, and the overall latency of the system from detection to user response with Maya. I will be performing the same analysis for the stairs hazard, with the addition of measuring how accurate the classification of the class stairs is. Note that I will not be testing the accuracy of specific object classifications because the response for different objects which pose as a hazard does not depend on what specific object it is, but on its overall position and size.
For hazard detection, I will perform an equal number of tests on large indoor items (such as tables and chairs), smaller items that should be detected (such as a laptop), and insignificant objects (such as flooring changes) to ensure false positives are not occurring. I will record true positives, false positives, and false negatives (missed hazards), aiming to achieve at least 90% true positive rate and no more than 10% false positive rate across these tests. I will also measure the latency from visual detection to haptic response with Maya, expecting a response time of less than 1 second for real-time feedback.
For stair detection, I will perform tests consisting of different staircases, single-step elevation changes, and flat surfaces used as negative controls (to ensure stairs are not falsely detected). Each group will be tested under varied indoor lighting and angles. The stair classification model will be evaluated on binary detection (stair vs. not stair). I aim to achieve at least 90% stair detection accuracy and 84% accuracy in distinguishing stairs from walls and other obstacles.

Posted on

Team Status Report 4/12

What are the most significant risks that could jeopardize the success of the
project? How are these risks being managed? What contingency plans are ready?

The risks that could jeopardize the success of the project is the battery not working for the Jetson Orin Nano and it possibly frying our Jetson. We performed extensive research on this to make sure it won’t fry it but for the small chance that it will, we backed up all of our code on github and have all of the individual components. working seperately

• Were any changes made to the existing design of the system (requirements,
block diagram, system spec, etc)? Why was this change necessary, what costs
does the change incur, and how will these costs be mitigated going forward?
• Provide an updated schedule if changes have occurred.
• This is also the place to put some photos of your progress or to brag about a
component you got working.

There have been no chances to the existing design. There have been no changes to the schedule.

We have finished composing our cane!

 

Now that you have some portions of your project built, and entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

Validation

We plan on testing all of the various main features individually and then together. This means testing the object detection on 50 various objects, testing the wall detection on 50 various walls, testing the FSR on 25 different surfaces. For the various object and wall tests, the haptic feedback should detect on 48 out of the 50 tests respectively (~95%). For the 25 different surfaces, we want the FSR’s to detect the floor on 24 out of 25 of them (~95%).

Additionally, we want the system to only have false positives <= 5% of the time.

Posted on

Maya’s Status Report 4/12

Accomplishments this week:
This week, I wrote the code for the FSR, and soldered all of the connections together which means our entire cane composition is mostly complete. This included everything for the haptics, and both FSRs and their connections with the QT Py. My code currently prints out when each pressure pad is pressed or not pressed, and we are currently working to integrate these responses with the Computer Vision.
The left image is the feedback we get with pressing and releasing the FSRs. The right picture is how all of our wiring is set up with the cane.

Reflection on schedule:
We had a lot of important progress this week, and everything other than our power supply has finally come together. According to our schedule, our entire cane should be complete by Wednesday, and we should begin completing usability testing, so I think we are on track with that as long as the barrel jack converter we ordered is the fix to our power issues.

  • Plans for next week:
    Over the next week, I will be working on the power supply, and we will begin our usability testing. We will also begin working on our final presentation and report!

    Verification:
    FSRs:

  • To verify the FSR system, I applied various pressures to the cane while logging voltage readings and observing whether they responded correctly to pressing and lifting.
    • A threshold voltage of 1V was chosen to distinguish between cane contact and non-contact, based on real-world walking pressure tests.
    • When the voltage exceeds the threshold, the QT Py sends a serial signal “ON” to the Jetson to indicate ground contact and trigger the computer vision script.
    • When the cane is lifted and pressure is removed, the QT Py sends “OFF”, and the Jetson pauses the object detection process to conserve resources.
  • I will also be testing the accuracy and responsiveness of this signal transition by walking with the cane and confirming that the system correctly activates only when the cane is placed on the ground.

    HAPTICS

  • The haptics send proper feedback based on which obstacle type is sent to it. We verified this by manually creating each object type and confirming the correct response was output.

    OVERALL:

  • The subsystem that deals with the QT Py was considered verified because it correctly detects cane contact, communicates with the Jetson, and produces haptic feedback reliably. We determined this was reliable because the feedback matches the print statements that we have on the screen based on object, wall, and stair locations.

 

 

Posted on

Team Status Report 3/29

Risks:
The only risk is that sometimes the depth stream from the LiDAR camera gives us the wrong data/doesn’t work in random holes of the frame, so it tells us objects are 0 meters away which will likely lower our accuracy but hopefully won’t end up interfering with too much.

Changes:
We changed our plan back to the original plan of using pyrealsense2.

Posted on

Cynthia’s Status Report 3/29

Accomplishments this week:
Finished debugging and integrated my fine-tuned YOLOv8 model that now classifies stairs along with objects (image included below — note that our Jetson has to be plugged in now, so we couldn’t bring it to actual stairs but I emulated stairs with the cart in the image).  Helped Kaya with fixing pyrealsense a little, then once it worked I used his code that gets distance from grid points to make the code that runs the model and creates bounding boxes now get the distance to the center points of the objects too.  I also worked with Maya to get the haptics working from the Jetson.  Lastly, I wrote the code to integrate/trigger the haptics and decide what action to suggest with the object detection model, based on objects detected, their location, and their distance — this currently works but the suggested actions are not all correct yet.

Reflection on schedule:
I believe I did a lot this week and caught up to what we planned.

Plans for next week:
Work with Kaya and get the wall detection working.  Fix the recommended action decision making code.

Posted on

Cynthia’s Status Report 3/22

Accomplishments this week:
I worked with Kaya after the correct versions of the libraries we needed were installed (after a lot of trouble and many hours spent on this) to get the pre-trained ML model working on the live RGB stream (see photos below).  I additionally had to change the code I previously wrote to work around what we decided does not work (pyrealsense2) on the Jetson, which we were depending on for its depth stream.  Recently, I started working with Kaya to get depth data and figure out how we can use that data in my python scripts instead of just getting it as a terminal command output.

Reflection on schedule:
I think I am slightly behind what our schedule was for my portion, but that is because we switched to using the camera on the Jetson earlier than planned since I am not able to use the library I need on the desktops and cannot use the camera on my laptop, so I was never able to test my code until one day ago and have not included stairs in our model yet.  Additionally, I was sick (and still am) and was able to work less than planned.  Overall, because of the rearranging, we are still on schedule as a group, but I need to continue to make good progress with our model moving forward.

Plans for next week:
Write code to train the model to incorporate a dataset of stairs and work further on getting distance measurements without pyrealsense2.

Posted on

Maya’s Status Report 3/22

Accomplishments this week:
This week, I setup the haptics and created a few sample patterns that we will be using with our Jetson.  The haptic patterns for each obstacle type is demonstrated here.

Reflection on schedule:
We were a bit behind schedule at the beginning of the week because we had some problems with the Jetson and L515 compatibility, but we put in a lot of hours this week to recover from this. Personally, that included me helping Kaya with the Jetson and L515 connections, and I also setup the haptics and created case statements for when each haptic pattern is set off.

Plans for next week:
Over the next week, I will be working to run the haptics over the Jetson and hopefully begin to connect the haptic responses through the CV code, and I will begin testing the overall power consumption to make sure it is under 30W and 5V.

Posted on

Maya’s Status Report 3/15

Accomplishments this week:
This week we linked the Jetson and the L515 using the SDK viewer, which is pictured below, and we are continuing to set up the L515 on the Jetson with our current code instead of the viewer code.

Reflection on schedule:
We are on schedule for the most part, but we have a heavy workday on Sunday to finish our goals for this week, which includes starting to integrate the haptics with the Jetson.

Plans for next week:
Over the next week, we will be working on the haptic logic and making sure it integrates with the Jetson.

Posted on

Team Status Report 3/8

Risks:

We have yet to attempt connecting the Jetson and L515, so that is a potential risk we may face, but we will be trying to do that this week so that we have ample time to problem solve if it does not work initially.

Changes:

The only change we have made is a new power supply due to our new power calculations. We did not realize that our computer vision would require our Jetson to be in Super Mode, which requires an additional 10W from what we had originally planned for. But we have found a new power source that supplies our required 5V, 6A.

A was written by Maya, B was written by Kaya and C was written by Cynthia.

Part A: Our cane addresses a global need for increased accessibility and independence for individuals with visual impairments. Around the world, millions of visually impaired people face mobility challenges that hinder their ability to safely navigate unfamiliar environments. The need for better mobility tools spans urban areas, rural villages, and developing areas, meaning it is not limited to any one country or region. Our design considers adaptability to different terrains and cultures, ensuring the cane can be valuable in settings from crowded malls to personal homes. By enhancing mobility and safety for people with visual impairments on a global scale, the product contributes to broader goals of accessibility, inclusivity, and equal opportunity.

Part B: Our cane addresses different cultures having varying perceptions of disability, independence, and accessibility. In communities with strong traditions of communal living, the single technology-advanced cane encourages seamless integration into these communities by drawing less attention and allowing users to maintain their independence.  Additionally, the haptic feedback system will allow for users to integrate seamlessly by drawing less attention by producing no noise from the device. By considering these cultural factors, our solution will allow for greater acceptance and integration into various societies.

Part C: We designed CurbAlert to take into consideration environmental factors, such as disturbing the environment around the user and interacting with the environment. Specifically, the feedback mechanism (haptic feedback) was chosen to notify only the user without creating extra noise or light or disturbing the surrounding environment or people. Additionally, our object detection algorithm is designed to detect hazards without physically interacting with the user’s environment and without having to be in contact with anything besides the ground and the user’s hand. Additionally, our prototype will be robust and rechargeable, making the product have no additional waste and making it so that a user will only need one of our prototype. By being considerate of the surrounding environment, CurbAlert is eco-friendly.

Posted on

Maya’s Status Report 3/8

Accomplishments:

This week, we found out that our Jetson would consume more power than we had initially planned for, so I spent a lot of time researching new power options that met our 5V, 6A power requirements of a portable charger. Kaya and I also worked to set up the Jetson Nano. Lastly, I did a lot of the final documentation and diagrams of our Design Review.

Progress:

We are on schedule now that we have finished the Design Report, our Jetson initialization, and L515 camera set up.

Future deliverables:

Our adafruit order was delivered, so I will be able to start working on the haptic vibration motor and starting to create the logic for different feedback patterns. Since Kaya and Cynthia will be working together on the software behind the computer vision, I plan to focus on more of the Jetson and haptics.