Ethan’s Status Report for 3/22

This week, I started full dataset training for the YOLOv8 OBB model. I implemented the loss function using three tunable hyperparameters that adjust the weight of the bounding box regression loss, the classification loss, and the angle loss. The intent of this was so to show more transparency in the unified loss calculation (the sum of all the previously mentioned loss), and if one loss was too high I could adjust how much the penalty would be. This method would hopefully allow us to better control the model’s convergence and allow for a better check pointing scheme that would save each best model in all four loss types (unified, regression, classification, and angle) to retrain later on. While the model started to train, I started working on creating a Docker image for the Jetson to make porting over the model easier.

Next week, working on some visualization code that would plot the predicted bounding box and class on static images in a clean manner (the core logic of this will be scaled out later for the Jetson)

Currently, I am on schedule.

Teddy’s Status Report for 3/22

This week was spent working on getting the vacuum end-effector to work. After some testing and design changes, I was able to get the end-effector to the point where it is able to pick up all 4 types of recycling consistently. I still need to add a release mechanism, which should just be a solenoid which opens and closes a hole in the air flow. I’ve designed and 3d-printed a part that should fit onto the end of the z-axis rack. I’ve also started looking into the stereo camera and the repository that comes with it to get the depth map from the camera. I also spent some time this week ordering parts, including the belt for the conveyor belt, silicone tubing for the vacuum pump, and aluminum extrusions for the gantry. The current construction of the gantry’s frame is hypothetically sufficient, but there seems to be severe wobbling, so switching from aluminum angles to aluminum extrusion should hopefully solve the problem.

Next week, I hope to attach the vacuum pump and suction end-effector to the gantry, get the conveyor belt cloth attached, get the depth map from the camera, and if time allows replace the aluminum angles with aluminum extrusion. I am behind schedule but I am catching up to what was planned.

Alejandro’s Status Report for 3/15

I spent the majority of this week working on reconnecting the Jetson to the webserver, since it no longer connects to my WSL server.js instance. I spent a considerable amount of time trying out different ports and connection methods. The first method I tried was to try and reset the firewall privileges on my laptop, but that failed to work. I then moved onto attempting to change the port forwarding on my device to the WSL port, but that also failed to work properly. The Jetson pings to my laptop but no longer wants to connect to the server. If I am unable to get the websocket working again as I had it before, we may have to shift to a tethered wired connection for the time being. Additionally, the canvas drawing feature is implemented. Also, the gantry xy axis is fully assembled now.

In lieu of this obstacle, I will plan to try one more time to get the websocket connection to work, and if it fails I will shift to using a wired connection. Additionally, I will test the canvas drawing with some test values next week as well.

I am currently on schedule aside from the minor hiccup.

 

Teddy’s Status Report for 3/15

Most of the time spent this week was on expanding and reinforcing the gantry to compensate for the sag caused by scaling up the design of the 4xidraw. I was able to construct a large frame around the conveyor belt in order to provide a mount for the 4xidraw. I then had to redesign some of the parts of the 4xidraw so that there would be no sag; this involved making one of the rods slide along the frame so that it supported the end-effector. I was also able to print out the rack so that the rack-and-pinion z-axis movement was complete. I’ve also begun testing the vacuum pump with the suction cup. It seems to be able to pick up flat, non-porous objects like plastic consistently, however it does not work as well with porous materials like paper.

Next week I plan to start working on programming the xy movement and will hopefully finish a fleshed out design of the vacuum end-effector. I am slightly behind schedule as I didn’t anticipate the rebuilding of the gantry to take as long as it did.

The picture of the current gantry is shown below:

Team Status Report for 3/15

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risks to our project is getting the machine learning model to real-time speed. In particular, the YOLOv8 architecture can achieve real-time speed on the Jetson Orin Nano however it has been noted it is incredibly frustrating to do so as it is careful speed and accuracy tradeoff analysis. To mitigate this risk, the model not being fast enough, we need to approach the speed and accuracy tradeoff analysis very meticulous, in particular we plan to have a detail log that we can reference to determine the sweet point for our use case.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

There are no current changes made to the existing design of the system.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Ethan’s Status Report for 3/15

This week I was able to get the YOLOv8 OBB model work on the simple single object toy images and multi-object toy image as described in the Design Report. To recap simple single object toy images, are image where there is a single object in frame that is relatively close to the camera and the multi-object toy images. Visually the bounding boxes and classification labels on objects in these images are accurate (± 5 pixels within the center of the true bounding box). With this, I think we are ready train the model on the full dataset and see how its purpose is.

Next week, if the results are good I plan on porting the model weights to the Jetson and seeing its performance there.

Currently, I am on schedule.

Team Status Report 3/8

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk to our project is that the suction-based end effector won’t be able to pick up different types of trash items. We’re managing this risk by focusing a considerable amount of time on the development of the end effector mechanism. In the case that the suction with the vacuum pump is ineffective, we plan to pivot to a claw like end effector mechanism for picking up the pieces of trash.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

There are no current changes made to the existing design of the system.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Part A was written by Teddy, Part B was written by Ethan, and Part C was written by Alejandro.

Part A:
with consideration of global factors. Global factors are world-wide contexts and factors, rather than only local ones. They do not necessarily represent geographic concerns. Global factors do not need to concern every single person in the entire world. Rather, these factors affect people outside of Pittsburgh, or those who are not in an academic environment, or those who are not technologically savvy, etc.

Currently, the issue of proper waste processing is one that many countries, including the US, are struggling with. Many countries do not have the infrastructure to pay for recycling facilities, as it requires a lot of manpower, and the resulting materials are not very profitable. Additionally, a lot of our waste is shipped to other countries in order to prevent it from piling up here, however this means that other countries are burdened with our trash. SortBot could help reduce the amount of trash by separating out the useful materials, at a much lower cost compared human workers. This could potentially help other countries without the funds for recycling management to do so.

Part B:

In the United States, the common practice is to dispose of all types of municipal solid waste in single bin, prioritizing convenience over environmental concern. SortBot is set out to challenge this norm, one that focuses on ease and efficiency, by introducing an autonomous system that streamlines waste separation without requiring a behavioral change. By shifting the responsibility of sorting waste away from the person throwing away the trash, SortBot can work within the existing American cultural norms. This ensures that materials can be property sorted without requiring individuals to change their habits. While this is not the best solution for this issue, it is one of the most appropriate for current American culture.

Part C:

Our product solution will meet the need of the consideration of environmental factors by providing a streamlined method to making the environment cleaner. Utilizing advanced computer vision and machine learning algorithms, our robot has the ability to identify and differentiate between different types of trash and can thus correctly categorize the trash types and sort them into their respective bins. The system can aid in the sorting of trash to aid in the recycling of these items. Normally, all recycling items mixed with trash would normally end up in landfills and be lost in terms of the value that they could provide when recycled, but now with our product solution, we can recover these items and help in reducing the environmental strain of waste disposal.

Ethan’s Status Report for 3/1

The majority of effort this week was spent on finding bugs in our YOLO codebase that caused the results from last week to look really poor. I discovered it was the loss function. Previously I was using a more naive approach that combined a weighted mean-square loss for the bounding boxes with a weighted cross-entropy loss for classification. After reading a couple of Medium articles about YOLO, I realized that I was implementing an entirely different loss function. Once I fixed that I also a little bit more training infrastructure that would hopefully making training analysis easier: curve plotting for each loss. By monitoring loss, I can identify parts of the model that would need tuning in future runs.

Next week, I plan on getting detection working on toy images.

Currently I am on schedule.

Teddy’s Status Report for 3/1

Over the last two weeks, I worked on constructing the gantry as well as the frame that holds it. I was able to construct the 4xidraw part of the gantry since all of our parts had finished arriving. I was then able to print, assemble, and attach the z-axis to the gantry. Currently, the parts required for the z-axis movement are assembled, but the end-effector is not attached, since we are still waiting for the vacuum pump and the relays to arrive. There is an issue in that the z-axis can’t extend very far away from the gantry, as the weight causes the rods to sag. In order to remedy this, I’m planning on attaching a bar horizontally onto the part where the z-axis attaches to the rest of the gantry, which will rest on two other bars so that the front does not sag. A picture of the current gantry is shown below.

I am a little behind schedule, partly due to delays in the shipping of the gantry parts as well as the end-effector parts. I plan to fix the sagging issue on the gantry next week, as well as hopefully get the electronics and the code for the 4xidraw working.

Alejandro’s Status Report for 3/1

I spent the majority of the week working on the assembly of the x-y axis of the gantry system. The assembly was time-consuming and laborious. The assembly requires a few more steps that will take some considerable amount of time. I will work on finishing the assembly the week I get back from Spring break. My progress is slightly behind schedule. In this case, I will prioritize the assembly of the gantry’s xy axis in the following week of work. Additionally, I was experimenting with the use of the canvas drawing onto the camera feed but I will need more time to solidify the feature. I’m also running a bit behind with the canvas drawing feature but I’ll have that done promptly after assembling the gantry. So next week I intend on completing the gantry xy axis and finishing the implementation of the bounding box HTML feature of mine.