Category: Status Reports

The majority of effort this week was spent on finding bugs in our YOLO codebase that caused the results from last week to look really poor. I discovered it was the loss function. Previously I was using a more naive approach that combined a weighted mean-square loss for the bounding boxes with a weighted cross-entropy loss for classification. After reading a couple of Medium articles about YOLO, I realized that I was implementing an entirely different loss function. Once I fixed that I also a little bit more training infrastructure that would hopefully making training analysis easier: curve plotting for each loss. By monitoring loss, I can identify parts of the model that would need tuning in future runs.

Next week, I plan on getting detection working on toy images.

Currently I am on schedule.

This week, I was able to finish the training infrastructure to train the YOLOv8 OBB model. Now that I am able to train the model, I need to employ some sort of verification strategy determine that the model was implemented correctly before I do full batch training. I decided on training the model one single basic image (I am defining basic as an image where the object is close up and on top of a distinct background). After training on this image for a significant number of epochs, I found that the detected bounding box was completely off. Currently, I believe that something went wrong with the model’s OBB detection head and spent a majority time this week trying to verify this assumption.

Next week, I plan on getting detection working on this toy image and hopefully training using the entire dataset and analyzing the results from there.

Currently, I am on schedule.

This week I spent time investigating web socket video streaming methods for a web app on a local host. I realized that Django would be too slow as it isn’t optimized for real-time streaming purposes as it is mainly meant for handling database/multi-user applications. Nonetheless, I decided to shift the framework to Node.js since it’ll provide lower latency in video streaming. I worked on writing the JavaScript file to set up the server backend. Additionally, I rewrote the index.html file to display counters of the types of trash collected, a camera feed box, and information on the types of trash rules we’ll be following. I also, reinstalled the OS on the Jetson so that my team and I would have access to it. I also setup RealVNC viewer so that I could use the Jetson without connecting it to a monitor. To make this work without RealVNC, we may need to order a Vesa cable.

I tried streaming an mp4 video but encountered some issues on the front end. I’m going to continue debugging this so as to be on track with the video streaming component of the web app for next week.

Additionally, I was supposed to assemble the XY axis of the gantry system this week but parts are taking longer than expected to arrive. Thus, I will continue as planned and take a day in the upcoming week to assemble the XY axis of the gantry system.

This week, I was able to finish the initial implementation of the YOLOv8 OBB model. Unfortunately, I found that the dataset I found on Roboflow last week is no in the format that YOLO models expect. Inside of having a (x, y, w, h, theta, class label) ground truth for each object in an image, the dataset actually has the following ground truth (bbox coordinate 1, bbox coordinate 2, bbox coordinate 3, bbox coordinate 4, class label). In order to finish this, I need to re-annotate the dataset. I plan on finishing a script by the end of tonight to fix this problem.

Currently, I am a little behind schedule. To remedy this, I plan on continuing to work on implementing the training infrastructure tomorrow and Monday. My goal is to start training the model by Tuesday. This way I will have sufficient time to (i) debug my model implementation and (ii) write data augmentations to artificially increase the amount of data.

Next week, I plan to have the model trained for a reasonable number of epochs to determine what optimizations I need to do on it for the best performance on the training dataset.

Most of the week was spend finding parts to purchase for the xy part of the robot gantry. Tried to keep costs low while not sacrificing too much in quality where it counts (e.g., spent more on the rods for guiding the movement of the end effector in the xy plane, since they needed to hold up to tighter tolerances).  Also had to do some small design changes to the 4xidraw design since our gantry will be significantly larger.  Additionally did some research on an adequate vacuum end-effector and possible ways that the gantry could have motion in the z-axis.

Link to the BOM for the xy part of the gantry is here: https://docs.google.com/spreadsheets/d/1N34-p-gZ5hg3E984jC0rKGtYfBRqveXMVGyWjgI6nZA/edit?usp=sharing

Plan for next week is to start designing the z-axis movement and the end effector, as well as to start 3D printing parts.

Currently everything is on schedule.

 

The majority of my time this week was spent ensuring the web app’s front end was set up. This involved a considerable amount of research in terms of determining the tools that would be used to build the web app. In setting up the website, I determined that I’d be using Django as the main Python framework for setting up the web app. It utilizes the MVT architectural design, which should provide sufficient capability for the web app. This would also allow for future additions of more interactive features such as controlling and interacting with the robot or counting the number of categorized items in each category(metal, plastic, paper, garbage). I also spent some time experimenting with React but concluded that it is unnecessary for the website’s streaming purposes. I’ve gotten the web app to currently work on local host which is sufficient for our project needs at the moment.

For next week, I intend to complete the web app’s backend capability by having the video streaming component working.

Currently, everything is on schedule.

The majority of time this week was spent on two efforts: verifying that the ±5 pixel expectation of the machine learning model was both not too strict and not to lenient and determining if the Jetson Orin Nano has enough compute for our needs. While evaluating the ±5 pixel expectation, I searched for trash datasets on both Roboflow and Kaggle and eventually settled one on Roboflow that I really liked. After visualizing images from the dataset with their oriented bounding boxes, their centroid, and 5 pixel circle around their centroid, I see that 5 pixels is a robust expectation to have.  Regarding the compute of the Jetson Orin Nano, the specifications say that it has 1.28 GFLOPs and  medium-sized YOLOv8-OBB model needs 208.6 FLOPs.  Even with a FLOP efficiency of 20%, the Jetson Orin Nano should have more than compute to run the model and potential any other assistive processes that strength the centroid calculation process.

Next week, for the first part of the week, I have to investigate a little more time figuring out if fine-tuning existing YOLOv8-OBB models would be better in our use case as opposed to training one from scratch. Moreover, I want to finish preparing the dataset for our use case (e.g. making the background white for the images, making transformations that affect the lighting of the images, and etc.)

Currently, everything is on schedule.