Month: April 2022

This week, we are mostly testing and optimizing based on the tests.

For depth image modeling, we visually examined the threat levels generated from the stereo frame of the depth camera. The disparity frame below is calculated at room HH1307.

But the threat level from our model shows that ground is not recognized as a whole due to perceived depth based on depth camera’s downward orientation.

With the addition of a filter based on a height threshold, we are able to achieve the effect of ground segmentation and thus more realistic threat level as shown below (the deeper the color the smaller the threat).

Beside tests revolving around accuracies such as the one above, we also looked into latency across every stage of the sensing and feedback loop. To make sure data sensed by depth camera and ultrasonic sensors model the same time instance, data rate from Arduino to Raspberry Pi is tuned down to match the depth camera.

During user testing, we found it hard to tell the three vibration levels apart and thus changed the number of threat levels from three to two to make the vibration difference more distinguishable.

We also made communication between Raspberry Pi and Arduino more robust by explicitly checking for milestones like “sensors are activated” in our messaging interface instead of relying on hand-wavy estimates of execution times. This marked the 49th commit in our GitHub repository. Looking at the code frequency analysis of our repo, it can also be seen 100+ additions / deletions are made every week throughout the semester as we iterate in terms data communication, depth modeling, etc.

Meanwhile, the physical assembly of electronic components onto the belt is completed as two batteries, the Raspberry Pi and Arduino board have been attached besides just ultrasonic sensors and vibrators. And we put in our last order for a USB to Barrel Jack Power Cable so that the depth camera can be individually powered by a battery as opposed to through Raspberry Pi.

From now until demo day, we will do more testing(latency, accuracy, user experience) , design (packaging, demo with real-time visualization of the sensor to feedback loop), and presentation (final poster, video, presentation).

This week, I have primarily focused on fixing physical connections and making sure that everything works fine. We now have a second battery that we plan to attach to the depth camera so we have more battery life to support both the camera and the raspberry pi. I have also planed out some exterial design, we will be cutting out a piece of black table cloth to cover up large electronic parts such as the raspberry pi, the arduino, the breadboard etc. Since our project is supposed to wearable, we will spend some time next week to make it visually attractive as well.

I am currently also working on a python visualization code for the belt. The python code will communicate with the raspberry pi on an usb chord using serial communication, and naively receive inputs and visualize the information. I have yet to decide what additional features I want to add to help us demo the belt, but at the moment I am planing to display the reading for the six sensors, the vibration level for the six vibration motors as a starting point. Passing in depth camera information might be difficult since frames would take a lot longer to process then numbers, and it might slow down the raspberry pi which is not what we really want.

Next week, I will be finishing up the python visualization, finishing up our external decorations, and conduct more testing to make sure that our project would perform well .

This week I tried using the error between feature pixels and their baseline correspondents to classify the alert levels.

However, the model failed to output different levels no matter what the depth camera points to. Then I worked with Ning and updated the model to classify alert levels based on hard depth thresholds.

Besides depth image modeling, I finalized the design of the final presentation and build more features of Bluetooth audio and running the raspberry pi main script at boot up.

This week, I have been focused mainly on restructuring the belt and reassembling the belt. This includes making a new protective cover for the ultrasonic sensors, reconstructing all of the circuits and checking all of the components out again to make sure nothing is wrong. Examples of assembly can be seen below:

Front of the belt and back of belt (Yellow space is reserved for inserting depth camera)

Sample of circuit assembly

We were able to run some tests, and all of the sensors are working correctly. Tmrw we will conduct tests with all modules installed.

As for next week, I plan to finish up the visualize palrts and add decoration to the belt. Covering up more of the underlying structures and adding design to fit the bat theme

This week I primarily worked on setting up the pipeline of the depth camera. The diagram below illustrates the pipeline.

I also added a column index-based lower boundary on the depth map due to the nature of stereo vision.

Next week I will be working with the team on the belt assembly and with Kelton on depth camera testings.

For this week, our team has been focused on making sure that the core algorithm ran in the raspberry pi would function as expected and correctly evaluate incoming risks. We are mainly focused on processing the information from the depth camera. For more detailed information on this, check Kelton’s blog.

The other aspect is the information from the ultrasonic sensors. We tested out our machine in a regular lab environment, to see how well it would respond. So far the stats from the ultrasonic sensors seem quite stable. We did not make much change to the core algorithm behind processing the sensor data, only a few minor bug fixes here and there. Overall, the prototype belt has met our minimal expectancy in a select settings. We have yet to fully test out its functionality, which we hope to focus more on next week.

For the physical belt assembly, we are currently waiting on new materials such as foam casing for the ultrasonic sensors, shorter female jumpers for rerouting the circuits, and tape covering for exposed wires. We will finalize our assembly next week.

In our final week before the presentation, we would rebuild the physical belt one last time to make sure there are no component failures and everything is well organized. Then we would test out our belt and make any minor adjustments as needed. We will also add a new feature to visualize our belt’s functionality so we can better display our product.  Our team has agreed to finish up other dues over the weekend so we can focus on testing throughout next week and especially over the weekends. So far we are expected to meet our goals before the presentation.

This week I implemented the new modeling method of using the narrower range of feature pixels from Oak-D’s feature tracker module to compute the mean squared error against the baseline. Additionally, the non-overlapping region in the frame between the two stereo cameras’ field of view is filtered out based on depth threshold.

Another minor update in the data processing pipeline is that the parsing in raspberry pi main is matched with the new data format of multiple ultrasonic sensor readings in one line of message.

Next week, I will test if the new mean squared error is interpretable and significant enough to detect ground-level obstacles from depth image. Meanwhile, I will explore the professor’s suggestion of somehow retrieving derivatives of pixels across time to calculate obstacle speed.