Category: Weekly Status Report

In the days before the demo, I modified the pixel to inch parameter to fit the size of the meat within the range of error. This value is hardcoded instead of dynamically determined because the meat is anticipated to be a fixed distance from the camera in the final product, so having fixed values instead of a reference point is better.

Before the demo, I was able to download the necessary libraries and files onto the Jetson, however I lacked enough adaptors to get the keyboard and the camera both working at the same time, so I was unable to get that working before the demo. I was able to get the necessary adaptors to get the Computer Vision working on the Jetson during Carnvial.

In the previous week, I combined the 3 different computer vision algorithms into one program that invokes all of them. Along the way, I found out the blob detection was not as effective as I thought under dim lighting conditions. As a result, I had to make modifications to the blob detection by changing several parameters regarding eroding excess lines.

You may have noticed the image is a result of an object recognition and not of an image cassification network. There are several reasons for this. One, the dataset we collected had multiple types of meat strewn together, and at that moment, our team realized that if someone placed multiple types of meat on a plate in front of the robotic arm, a classification system is not robust enough to know that and may cause undercookng of meat. Another reason is that if the blob detection fails to work in a way we want, object recognition algorithm is our backup mitigation technique. While object recognition is slower than blob detection, it’s still sufficiently fast enough for our desired metrics.

What you see below is the result of YOLOv5 trained on 150 epochs on a tiny data set (only 20 images) augmented to be 60 images in total. The batch size each epoch was 12 images each. YOLOv5 was selected due to its speed advantage and its active community support online.

Currently on track to complete by the completion date indicated on the schedule, which is Monday, at least I am mostly done. The hesitation is because the integration period will provide a chance to add to the dataset, and that would require more training on the network.

By next week, I hope to begin the integration of the subsystems by having the files uploaded onto the Xavier. Hopefully that would lead to an improvement in detection time also.

As promised last week, I was able to solve the issue of blob detection not working on larger blobs. As it turns out, the reason why I was unable to solve the issue earlier was because I was turning off filtering mechanisms one by one, however, I actually had to disable multiple preset parameters to get the blob detection to work on larger images. Furthermore, this past week I was working on ways to improve the blob detector and edge detector on different types of images. As seen in last week’s report, back then I was only testing on a simple slab of meat on a white background. This week, I obtained more images, especially the type of image seen below where the meat is on a tray, which is how I expect the final environment to work. And as you can see, my changes in parameters means edge and blob detection work in more scenarios accurately (note the 3 circles for the 3 trays of meat).

Next week, my 2 primary tasks involve beginning the setup of a neural network and getting the blob and edge detectors to take in a live camera feed to test its effectiveness in a more real world setting. I anticipate the collection and tagging of a dataset.