Over this past week, I have been working on the bluetooth transmission system between the Jetson and the iPhone, as well as mounting the entire dirt detection system. I have also been working on tuning the dirt detection algorithm, as the algorithm I previously selected operated solely on static images. When working with the entire end-to-end system, the inputs to the Jetson camera are different than the inputs I used to test my algorithms. This is because the Jetson camera is limited in terms of photo quality, and it is not able to capture certain nuances in the images when the vacuum is in motion, whereas these details may have been picked up when the system was at rest. Moreover, the bluetooth transmission system is now fully functional. I obtained a 800% speedup; I previously was working with a 45 second average delay between messages. After calibrating the Jetson’s peripheral script, I was able to reduce the delay between messages to be just 5.5 seconds (on average). This is still not as fast as we had initially hoped, but I recognize that running the dirt detection script will incur a delay, and that it may not be able to granulize this window any further. I am still looking to see if there are any remaining ways to speed up this part of the system, but this is no longer the main area of concern which I am working with.
Additionally, prior to this past week, the BLE system would often cause Segmentation Faults on the Jetson’s side when terminating the script. This was a rather pressing issue, as recurring Segmentation Faults would cause the Bluetooth system to require reboots at a regular rate. This is not maintainable, nor is it desired behavior for our system. I was able to mitigate this issue by refactoring the iPhone’s bluetooth connection script. Prior to this week, I had not worked too extensively with the Swift side of things; I was focused more on the Python scripting and the Jetson. This week, I helped Nathalie with her portion of the project, as she had heavy involvement with the data which the Jetson was transmitting to the iPhone. After a short onboarding session with my group, I was able to make the cleanliness data visible from the AR mapping functions, rather than only within scope for the BLE connection. Nathalie now has access to all the data which the Jetson sends to the user, and will be able to fully test and tune the parameters for the mapping. We no longer have any necessity for backup Bluetooth options.
Beyond the software, I worked closely with my group to mount the hardware into the vacuum system. Since I was the one who performed the tests which determined the optimal angles for which to mount the camera, I was the one who ultimately put the system together. Currently, I have fully mounted the Jetson camera to the vacuum. We will need to affix the active illumination system alongside the Jetson camera which I am not too worried about. The new battery pack we purchased for the Jetson has arrived, and once we finish installing WiFi capabilities onto the Jetson, we will be able to attach both the battery system and the Jetson itself to the vacuum system.
I have since started running some tests on the end-to-end system; I am currently trying to figure out the bounding boxes of the Jetson camera. Since the camera lends itself to a bit of a “fisheye” shape, the image that we retrieve from the CSI camera may need to be slightly cropped. This is easily configurable, and will simply require more testing.
Over the next week, I intend to wrap up some testing with the dirt detection system. The computer vision algorithm may need a little bit more of slight tuning, as the context in which we are running the system can highly change the performance of the algorithm. Moreover, I hope to get WiFi installed onto the Jetson by tomorrow. This is causing issues with the BLE system, as we are relying on UNIX timestamps to match location data from the AR app with the Jetson’s output feed. Without a stable WiFi connection, the clock on the Jetson is constantly out of sync. This is not a hard task, and I do not foresee this causing any issues in the upcoming days. I also plan to aid my group members in any aspect of their tasks which they may be struggling with.
