Teddy’s Status Report for 4/12

I spent the last two weeks overhauling the entire physical design of the gantry. I replaced the aluminum angles with aluminum extrusions, and used brackets with t-nuts to provide an easily adjustable yet sturdy frame. I made it so that the supporting rod now goes through two wheel attachments which slide across the rails instead of having the rod sit on the edge of the frame. The z-axis rack has been changed from a 3d-printed part to a lazer cut one in order to make sure that it is rigid and straight. I also abandoned the bearing wheels on the z-axis in favor of a 3d-printed guide slot. I replaced some parts of the conveyor belt in order to add tension, and was able to pull the belt taught. I also made a mount for the conveyor belt stepper and added the stepper and the timing belt onto the conveyor, so that the conveyor now moves with the stepper motor. In short, the physical components of the gantry are essentially finished.

Next week, I plan to try to get the depth information from the stereo camera and help work on all of the steps necessary for integrating the work done by my teammates. I am behind schedule but I believe we should get everything done in time.

Verification

For our testing, we will test the gantry’s ability to move with a certain granularity. We’ll move it a certain number of steps and measure its movement in order to determine what the distance corresponding to a step is. We’ll also be doing rigorous testing with multiple common trash/recyclable items such as bottles, cans, cardboard, jars, chip bags, etc. to ensure that the end-effector is able to handle a wide variety of materials and surfaces. We will also make sure to test objects with different weights to determine if it is able to lift objects up to 1 lb. We’ll also be testing the overall accuracy of the depth information (z-coordinate of the object) calculated from the stereo camera.

Alejandro’s Status Report for 4/12

This week I mainly focused on getting the path sequencing to work with the motors. I changed the code in Arduino to run on a switch case statement control flow with four main states. There are some bugs with this as when the motors are told to move to a specific trash bin coordinate they move to the origin, which is wrong. Additionally, I implemented the button and the speed control frontend and backend on the web app portion of our system. Now I just need to ensure that the Arduino can take the data relayed from the Jetson and execute it correctly. I also got the web app to receive the bounding box coordinates from the Jetson.

As for the verification of the measurements, I intend to schedule the pick up and drop sequence without suction. My analysis will be based on a scale of time. If the gantry is able to reach a location within 15 seconds and complete the entire execution of pickup and drop off within that time (excluding the suction) then I know that the path sequencing component works as per the design requirements on my end. Additionally, I will test the latency of the web app video feed by running a timestamp on the jetson when it sends data to my server and then timestamp the receipt of it on the server. Then I will perform the analysis by taking the difference of the two times to see if the latency meets the use case requirement.

My plans for the future are to fix the path sequencing code and finalize the canvas drawing on the front end of the web app. I also plan on coding the controls for the conveyor belt. I also need to ensure that the Arduino receives the commands from the Jetson’s relay of commands from the web app interface.

I am currently on schedule.

Team Status Report for 3/29

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

One of the most significant risks currently is that the stepper motors will not have enough speed to complete the pickup sequence in 8s. We have had to run the gantry slower than the maximum setting as the frame currently isn’t well-built enough to handle the stresses. We plan to address this by changing the frame to using aluminum extrusions instead of aluminum angles. One other risk is that the stereo camera might not be accurate enough to sense the depth for accurate placement of the end-effector on the trash items. Currently our contingency plan for this is to use a pressure sensor to detect when the end-effector touches the object.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

Other than the change to the frame mentioned above, there are no planned changes to the existing system. This change was necessary as parts of the gantry were bending and buckling during the gantry’s movements. Replacing the aluminum parts costs around $60, which is not much of a problem since after the purchase we still have around $250 of budget remaining.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Ethan’s Status Report for 3/29

This week, I worked on setting up the Jetson Orin Nano. I currently was only able to setup PyTorch after a long couple of days because I hit a couple of roadblocks with setting up CUDA and cuDNN, Most of my issues stemmed from incorrect paths that weren’t properly set in the environment variables and incorrect versions downloaded by “pip3 install torch”.  Unfortunately most of progress was wiped as I accidentally did something to the drivers (I have no idea what because I was levels deep in random Nvidia documents). In a better light, I also wrote some code for Monday’s demo.

Currently, I am behind schedule a little. I plan to fix this by continuing to work the Jetson Orin Nano for the rest of this week and early next week.

Ethan’s Status Report for 3/22

This week, I started full dataset training for the YOLOv8 OBB model. I implemented the loss function using three tunable hyperparameters that adjust the weight of the bounding box regression loss, the classification loss, and the angle loss. The intent of this was so to show more transparency in the unified loss calculation (the sum of all the previously mentioned loss), and if one loss was too high I could adjust how much the penalty would be. This method would hopefully allow us to better control the model’s convergence and allow for a better check pointing scheme that would save each best model in all four loss types (unified, regression, classification, and angle) to retrain later on. While the model started to train, I started working on creating a Docker image for the Jetson to make porting over the model easier.

Next week, working on some visualization code that would plot the predicted bounding box and class on static images in a clean manner (the core logic of this will be scaled out later for the Jetson)

Currently, I am on schedule.

Teddy’s Status Report for 3/22

This week was spent working on getting the vacuum end-effector to work. After some testing and design changes, I was able to get the end-effector to the point where it is able to pick up all 4 types of recycling consistently. I still need to add a release mechanism, which should just be a solenoid which opens and closes a hole in the air flow. I’ve designed and 3d-printed a part that should fit onto the end of the z-axis rack. I’ve also started looking into the stereo camera and the repository that comes with it to get the depth map from the camera. I also spent some time this week ordering parts, including the belt for the conveyor belt, silicone tubing for the vacuum pump, and aluminum extrusions for the gantry. The current construction of the gantry’s frame is hypothetically sufficient, but there seems to be severe wobbling, so switching from aluminum angles to aluminum extrusion should hopefully solve the problem.

Next week, I hope to attach the vacuum pump and suction end-effector to the gantry, get the conveyor belt cloth attached, get the depth map from the camera, and if time allows replace the aluminum angles with aluminum extrusion. I am behind schedule but I am catching up to what was planned.

Teddy’s Status Report for 3/15

Most of the time spent this week was on expanding and reinforcing the gantry to compensate for the sag caused by scaling up the design of the 4xidraw. I was able to construct a large frame around the conveyor belt in order to provide a mount for the 4xidraw. I then had to redesign some of the parts of the 4xidraw so that there would be no sag; this involved making one of the rods slide along the frame so that it supported the end-effector. I was also able to print out the rack so that the rack-and-pinion z-axis movement was complete. I’ve also begun testing the vacuum pump with the suction cup. It seems to be able to pick up flat, non-porous objects like plastic consistently, however it does not work as well with porous materials like paper.

Next week I plan to start working on programming the xy movement and will hopefully finish a fleshed out design of the vacuum end-effector. I am slightly behind schedule as I didn’t anticipate the rebuilding of the gantry to take as long as it did.

The picture of the current gantry is shown below:

Ethan’s Status Report for 3/1

The majority of effort this week was spent on finding bugs in our YOLO codebase that caused the results from last week to look really poor. I discovered it was the loss function. Previously I was using a more naive approach that combined a weighted mean-square loss for the bounding boxes with a weighted cross-entropy loss for classification. After reading a couple of Medium articles about YOLO, I realized that I was implementing an entirely different loss function. Once I fixed that I also a little bit more training infrastructure that would hopefully making training analysis easier: curve plotting for each loss. By monitoring loss, I can identify parts of the model that would need tuning in future runs.

Next week, I plan on getting detection working on toy images.

Currently I am on schedule.

Ethan’s Status Report for 2/22

This week, I was able to finish the training infrastructure to train the YOLOv8 OBB model. Now that I am able to train the model, I need to employ some sort of verification strategy determine that the model was implemented correctly before I do full batch training. I decided on training the model one single basic image (I am defining basic as an image where the object is close up and on top of a distinct background). After training on this image for a significant number of epochs, I found that the detected bounding box was completely off. Currently, I believe that something went wrong with the model’s OBB detection head and spent a majority time this week trying to verify this assumption.

Next week, I plan on getting detection working on this toy image and hopefully training using the entire dataset and analyzing the results from there.

Currently, I am on schedule.

Teddy’s Status Report for 2/22

The bulk of the time this week was spent designing parts for the z-axis movement/end effector as well as working on making a functioning conveyor belt. I’ve created CAD models for the rack and pinion movement, and have started assembly of the parts. I received multiple parts that had been ordered, including the suction-cup attachment for the end effector, and after some testing, decided that it would be necessary to actively pull a vacuum instead of having just passive suction. After some research, I was able to find a way to pull a vacuum (with minimal noise so it doesn’t interfere with other demos) and have ordered the parts for the vacuum attachment. I’m currently working on designing the 3d printed parts that will attach the vacuum and the end effector to the rack and pinion system. Additionally, I was able to construct a frame for the conveyor belt out of 3d printed parts and aluminum (image of frame is shown below). Currently working on designing a basic belt that will be the surface that the trash will rest on.

I am currently a little behind schedule, as the switch to an actively-pulled vacuum requires additional design work. I’m planning on finalizing the design of the z-axis/end effector and working on the Arduino code for the gantry xy movement for the over the next week.