Posted on

Ishan’s Status Report for 3/30/2024

This week I fixed some bugs with regards to the code running the distance and camera sensors to add a preview feature that would allow us to see a preview of the photo of the object that the user would take. Additionally, I worked on building the serial UART interface between the RPi and NVIDIA Jetson which we will test tomorrow. This will hopefully allow us to transfer image data and distance data to the Jetson where it will be processed.

I believe we are currently on schedule.

Next week, I will continue to work on making sure the integration process is smooth for our device. I will also work on running latency tests for our product to ensure that the latency is at an acceptable level for our users. Furthermore, we will run tests with our cameras and sensors with our whole device to ensure that its functionality is consistent with what we want for our users.

Posted on

Ryan’s Status Report for 03/30/2024

This week, I collected some new data directly from one of our testing environments by taking pictures of trashcans, stop signs, benches, storefronts, cars, etc. This will help in the new model training process. I have begun training a new model for the moment. In addition, I have also created a flask server in our Jetson to take in input from the raspberry pi, host the model, and run the input through the model to produce an output.

My progress is still on schedule.

Next week, I hope to get a much better model and incorporate the new model into the Jetson prior to our Demo. I also hope to start making the headset.

Posted on

Oi’s Status Report for 3/30/2024

What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

For this week, I spent a lot of time fixing up the design of the Nav-Assist iOS app and gathering feedback from people regarding on what features to have, particularly with the text-to-speech. I am leaning towards not having the user tap the screen for the app to repeat what’s on it, as it might conflict with the existing screen readers for visually impaired users. The next thing I worked on was learning how to integrate the app with Raspberry Pi in order to receive data from Ishan’s component. I looked into and analyze different ways to send data between the different components via Bluetooth. I also met up with Ryan and Ishan on connecting our parts together in order to prepare for the interim demo.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I believe my project is currently on schedule.

• What deliverables do you hope to complete in the next week?

For next week, I hope to be able to integrate and receive data from Ryan and Ishan’s components. I also hope to expand the UI more, as we get more data coming from the other components. I hope to make the UI smoother for the users.

Posted on

Team Status Report for 03/23/2024

What are the most significant risks that could jeopardize the success of theproject? How are these risks being managed? What contingency plans are ready?

Currently there are no major risks that could jeopardize the success of the project. As of now the biggest concern and challenge for all of us is how to integrate everything together and ensure that all the systems work smoothly together. In order to manage these risks, we are all trying our best in our own parts and testing it in a way that simulates the connection to the other parts.

 

• Were any changes made to the existing design of the system (requirements,block diagram, system spec, etc)? Why was this change necessary, what costsdoes the change incur, and how will these costs be mitigated going forward?

The iOS app now allows the user to tap on the screen, and it will read to the user the message on the screen. This allows the user to hear a repeat of what screen they are at as well as the state of the app, which will promote safety. There are no costs for this change.

No schedule changes have been made as of now.

Posted on

Oi’s Status Report for 3/23/2024

What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

For this week, I spent a lot of time reading about ethics and discussing about ethics with other teams. I’ve spent a lot of time in coming up with discussions and also contemplating about how to incorporate what I’ve learned from the discussions and research into the project. I’ve been thinking about how to make the design of the iOS app more accessible to users by changing and adding more features into the iOS app. I’ve been able to enable the app to read the screen to the users as well as also allow the user to tap on the screen they’re currently at, and the app will just read out what screen it’s at to the user. I struggled with this a little bit initially, as I had to find a way to do this without using the View Controller and do it directly through the Swift UI. This will be a great help to visually impaired users, as they will learn what stage the screen is  at.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I believe my project is currently on schedule.

• What deliverables do you hope to complete in the next week?

For next week, I hope to get the features incorporated on all pages, as well as adding images, but I’m saving it for the last, as that should be done pretty simply. I am also thinking about how to make the transitions between the notifications smoothly for the user. I also hope to finish the user guiding steps.

Posted on

Ishan’s Status Report for 03/23/2024

This week I continued the integration process by purchasing the portable battery supply and camera and conducted some testing on the length of the battery supply and how all the components interact together. I ran into a small problem with the RPi HQ camera we got from inventory as I didn’t realize that it needed an additional lens. So, I sent an order for the RPi camera module 3 which doesn’t require an additional lens and has a similar installation process to the HQ camera. This should be a relatively smooth adjustment process Overall, the portable battery, camera, and sensors are working well with the RPi and are producing the expected results.

I am currently on schedule.

Next week, I would like to run more testing on how our device functions with the integrated ML model to the RPi. Additionally, we will begin building the actual headset for the device, so we will then work on testing how the sensors and cameras operate when placed on the headset and with a user using it to maneuver.

 

Posted on

Ryan’s Status Report for 03/23/2024

This week, I finished the first round of model training. The accuracy was around 72%, much lower than expected. So, I have begun training the model again with some tweaked parameters and for more epochs. I also placed the order for the camera module, and have begun researching how best to incorporate the model into the Jetson.

My progress is still on schedule.

Next week, I hope to get a much better model and incorporate the model into the Jetson prior to our Demo.

Posted on

Oi’s Status Report for 03/16/2024

What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

For this week, I was mostly sick, so I could not come to class and was traveling on Monday, so I missed class. However, I was able to feel better soon and was able to do some work. I worked on the Ethics assignment and read about how technology can be political. While doing the ethics assignment, I’ve came up with different ways I am considering to add to my design as stretch goals, such as speaking in different languages to also reach more users. For the iOS app, I worked on integrating the text to speech feature onto the app, to read message off of the screen to the user. Originally, I was struggling a little bit to do that as it seemed to be that the iOS version had to be downgraded for that to work. However, after a few days of tinkering and considering different modules (I was considering using OpenAI too), I was able to get it working with Av Foundation! Currently, the app can read screens and the different states of the system to the user and also verify the user’s activity, such as when trying to connect to a peripheral bluetooth device.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I think I am a little bit behind, as I got sick and was stuck for a bit with the iOS problem, but for next week, I will try to ramp that up. I will be planning on working for more hours next week to compensate for being sick this week and also travel issues.

• What deliverables do you hope to complete in the next week?

I hope to finish setting up all the state screens and get the app to correctly read them to the user and also add images to those screens. I also hope to think about ways to interactively teach the user how to use the app through a tutorial, but I have to decide first what is to do. If not, I plan to start working on integrating the app with the Jetson and try to get basic data from it and learn more about data processing here.

Posted on

Ishan’s Status Report for 03/16/2024

This week I continued to work on integrating our device by working on implementing 6 sensors. Furthermore, I started working on interfacing the RPi with the Jetson Nano so that we can send information across both devices.

I am on schedule.

Next week, I’ll continue the integration process by starting to build the actual headband and placing the sensors and camera on the headband. Additionally, we’ll start to integrate the Jetson Nano with our ML code so that we are ready to formally test our equipment.

Posted on

Ryan’s Status Report for 03/16/2024

This week, I have been working on training my model for obstacle detection. I encountered a few errors while training, but the training is fully underway, and I hope to have some results by mid-next week. I have also finalized the RPi Camera Module V2-8 for our camera and will be placing the order before class on Monday.

My progress is back on schedule.

Next week I hope to finish the model and incorporate it into our Jetson. I also hope to use the camera to analyze the quality of our video and make tweaks to the model as needed.