My work this week focused on the Jetson. I got the camera connected to the Jetson and got it to capture images through terminal commands to gstreamer that save an image. I could run this command from a python file using os.system(). There is also a way to use opencv with a gstreamer pipeline to capture the image, but I haven’t gotten it to work yet. I will focus on other things for now, but the terminal command takes a bit longer since it has to set up and tear down the gstreamer pipeline every time, and the image seems to come out slightly orange, but we can at least capture images.
I also got the necessary dependencies to run pretrained yolo models on the Jetson with GPU acceleration. The dependencies were more complicated than I thought, with for example installing TensorRT (NVIDIA library for GPU-accelerated inference) required choosing the right installation for your Python and CUDA versions, but it worked out. After some basic testing, it seems like the system can perform inference on a jpg in ~50 ms, which should be more than fast enough.
Next steps are to train a YOLO model on our custom dataset. I found a dataset of recyclable waste, and split it into train/test/validation, and now that the dependencies are all set, we should be able to use the Jetson’s GPU to train.
Progress is on track.