Author: ikotecha

This week I worked on integrating the Bluetooth connection from the phone to the RPi using bluez, but we ran into some issues with compatibility with the ios applications, so we had to pivot to using an MQTT server instead which we worked on setting up. Additionally, I conducted further testing for the ultrasonic sensors and raspberry pi camera quality against the model to test if it works in different environments.

I believe I am on schedule.

For the next week, we will continue fine-tuning our MQTT server to establish a connection between the iOS application and the RPi as well as continue further testing in preparation for the demo.

• What are the most significant risks that could jeopardize the success of the
project? How are these risks being managed? What contingency plans are ready?

The biggest risks of the project are on the latency of our product and trying to keep computation time lower so that our latency is low enough for the user to effectively use. One change we’ve made is removing the Jetson as the data transfer from the RPi to the Jetson heavily affected our latency, so we decided to remove the extra device and do all the computation on the RPi. We have contingency plans and that’s to lower our data set that we will training our model on to potentially reduce latency even further if necessary for the demo.

• Were any changes made to the existing design of the system (requirements,
block diagram, system spec, etc)? Why was this change necessary, what costs
does the change incur, and how will these costs be mitigated going forward?

We have changed our existing design by removing the Jetson, so it will just be the RPi that will run our ML model as well as our distance code and we will send all that information from the RPi to the App. This change was necessary as we struggled to effectively use the Jetson, and we found that latency would be an even bigger issue if we were to use the Jetson as well as the RPi due to the time it would take to transfer data from the RPi to the Jetson

• Provide an updated schedule if changes have occurred.

We have no updates for our current schedule.

For this week, I focused mainly on developing the final presentation that I’ll be giving, so which means preparing the slides as well as what I’ll be saying for the presentation. Additionally, I focused on gathering all the information from the testing and validation that we conduct for all our individual components and for our system as a whole.

I am currently on schedule

Next week, we will continue to test our entire product by putting it through some more latency tests and testing it against different objects we will be assessing during our demo.

As you’ve designed, implemented, and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

We recognize that there are quite a few different methods (i.e. learning strategies) for gaining new knowledge — one doesn’t always need to take a class, or read a textbook to learn something new. Informal methods, such as watching an online video or reading a forum post are quite appropriate learning strategies for the acquisition of new knowledge.

For almost all tasks I completed, I had to learn various new skills, particularly pertaining to the Raspberry Pi, Ultrasonic sensors, and Pi Camera modules. Setting up the Raspberry Pi was extremely tough, especially connecting it wirelessly using the headless setup. I picked up several new strategies to acquire new knowledge among them was looking at YouTube tutorials or more visual tutorials to complete my tasks as seeing another person deal with the Raspberry Pi was far easier to follow than any documentation. Additionally, I found forums such as the Raspberry Pi forums very helpful to see for bugs and any potential issues as almost always there was another person who had a similar issue, and they were fixed on those pages. Finally, documentation was also very helpful especially for the Camera Module as I relied on it heavily to follow to set up the module, download the necessary dependencies, and so on.

 

This week I mainly focused on continuing to develop without the Jetson while we wait for it to be up and running. Firstly, I set up the RPi with CMU Wifi via a wired connection, but now that I’ve set up the initial connection, it’s easier to set up the wireless connection. Furthermore, I’ve worked on tweaking my detection model to account for objects that are entering the sight of one sensor to another to ensure that only one of the sensors is picked for the direction that will be sent to the Jetson. For this, I simply added to my filtering algorithm to ensure that there was an object detected on two adjacent sensors within a certain distance range and time frame then the measurement on the sensor furthest away from the user’s north will be discounted. Furthermore, I established Bluetooth connectivity from RPi to the IOS app, but this connection will be used secondary to the Jetson connection as this is a backup option in case of failure of the Jetson.

I’m currently on schedule.

Next week, we hope for the Jetson to be up and working so we will look to connecting the Jetson to the IOS app as well as ensuring the serial connection between the RPi and Jetson works as expected. Furthermore, we are going to build the actual device that the user wears so we can run tests based on how the product will look on the actual user.

Verification and validation:

So far I have completed thorough testing on the range and direction coverage of the ultrasonic sensors. I’ve completed individual tests of the ultrasonic sensors to see their degree of coverage and their distance measurement capabilities, and I have completed this same test with multiple sensors. In addition, I aim to complete further testing with multiple sensors but also analyzing how the sensors react if an object overlaps between two sensors or if an object goes from one sensor to another. These results will be analyzed to deduce the placement of the sensors on the headband and how much distance there should be between each sensor. In addition, I have used this data to adjust how my code is set up in regards to double detection on the sensors and to ensure there are no overlaps in how the objects are detected to ensure the user has the right bearings of the direction of the incoming object.

As far as testing the capabilities of the camera, I have completed testing based on how the camera works in different environments and if the photo quality is good enough for our ML model to see and run on. Furthermore, I will also complete latency testing for how quickly the data is transferred from the RPi to the Jetson and if it meets our latency requirements. Finally, I have also run tests on the portable battery to ensure that the battery life meets our requirements, and with my program running, there’s a battery life of 4 hours however I anticipate that will go down by the time we complete testing with running the entire model.

For the rest of the project, most of my testing will be completed with the actual device on the headband as results could differ when it’s placed on the user’s head. The testing I will complete for the physical headband will be similar to the testing I have completed on the individual sensors but account for the user’s head movement and body movement in general.

This week I fixed some bugs with regards to the code running the distance and camera sensors to add a preview feature that would allow us to see a preview of the photo of the object that the user would take. Additionally, I worked on building the serial UART interface between the RPi and NVIDIA Jetson which we will test tomorrow. This will hopefully allow us to transfer image data and distance data to the Jetson where it will be processed.

I believe we are currently on schedule.

Next week, I will continue to work on making sure the integration process is smooth for our device. I will also work on running latency tests for our product to ensure that the latency is at an acceptable level for our users. Furthermore, we will run tests with our cameras and sensors with our whole device to ensure that its functionality is consistent with what we want for our users.

This week I continued the integration process by purchasing the portable battery supply and camera and conducted some testing on the length of the battery supply and how all the components interact together. I ran into a small problem with the RPi HQ camera we got from inventory as I didn’t realize that it needed an additional lens. So, I sent an order for the RPi camera module 3 which doesn’t require an additional lens and has a similar installation process to the HQ camera. This should be a relatively smooth adjustment process Overall, the portable battery, camera, and sensors are working well with the RPi and are producing the expected results.

I am currently on schedule.

Next week, I would like to run more testing on how our device functions with the integrated ML model to the RPi. Additionally, we will begin building the actual headset for the device, so we will then work on testing how the sensors and cameras operate when placed on the headset and with a user using it to maneuver.

 

This week I continued to work on integrating our device by working on implementing 6 sensors. Furthermore, I started working on interfacing the RPi with the Jetson Nano so that we can send information across both devices.

I am on schedule.

Next week, I’ll continue the integration process by starting to build the actual headband and placing the sensors and camera on the headband. Additionally, we’ll start to integrate the Jetson Nano with our ML code so that we are ready to formally test our equipment.

This week a large portion of my time was spent working on the design document and report. I worked mainly on the system architecture, design tradeoffs, and system implementations where I made sure to detail the exact specs of our device as well as the other options we considered for different hardware components of our device. Beyond the design report, I ordered the Jetson and worked on setting up the Jetson as well as connecting it to the Raspberry Pi. Furthermore, I completed research on our prioritization algorithm and now have an outlook on how we’ll be filtering the data received by the ultrasonic sensors.

My progress is on schedule.

Next week, I hope to implement my filtering algorithm as well as make sure all the components of the device: sensors, RPi, and Jetson function as desired together.

This week I worked on implementing multiple sensors with the Raspberry Pi after getting all the materials like the SD card, sensors, and other connecting devices. I ran into some issues with inconsistent measurements from the sensors, so I’ve been looking into ways to mitigate these issues or if it would be better to look at other devices like an Arduino potentially.

I am a little behind schedule as I need to start working on the Jetson and interfacing it with the Raspberry Pi, but I still have time to catch up before spring break. I’m going to order the Jetson by tomorrow and begin working on connecting the Jetson with the Raspberry Pi so that we’re ready to transfer data between the two devices. This shouldn’t be too difficult to complete as I’ve done the necessary research, so I’ll know how to proceed once I get the device.

Next week, I hope to complete the connection between the Raspberry Pi and Jetson as well as work out the inconsistent measurements of the ultrasonic sensors. Finally, I aim to begin researching filtering/prioritizing algorithms that we would run with our object detection model so that our device will prioritize objects that are in immediate danger to our users.