This week, I worked on integrating the ML object detection with the hardware components that Will has set up. Most of the work that needed to be done was in the context of porting the model to the Jetson, then ensuring that the models themselves ran on the Orin GPU. This was a lot trickier than I anticipated, since Jetson Nano configurations with the underlying CUDA kernels aren’t as straightforward as using GPUs on the computing cluster that I had developed and initially evaluated the models on.
I was able to eventually get the object detection models running using a Docker container, which are custom modular builds that enable the usage of Jetson-specific GPU functionalities that are needed for the object detection models. After this started working, I observed a drop in inference time from 8 seconds/frame to 90 milliseconds/frame, so I am reasonably sure that this is working now.
I was also able to draft an implementation of a speech to text feedback loop in the navigation submodule, and I’ll be able to fully test it in the next few days to make sure the feedback is clear and timely.
With regards to progress, I believe that I am about on schedule. I am not particularly concerned about not being able to complete the necessary deliverables on my end, and in the coming weeks, I anticipate spending time working on the integration, possibly helping troubleshoot the other components when I can, which I started doing this week.
The next week, I also anticipate refining the navigation logic further, and getting a speech to text feedback loop tested on the Jetson, as mentioned before.