Posted on

Teddy’s Status Report for 4/26

This week was spent putting some of the final parts of integration in place with Ethan and Alejandro. I was able to rebuild the end-effector so that it is able to lift objects up to 1lb, by using a worm gear and some other gear ratios. I was also able to add a valve to the vacuum such that we can open a hole where the air can escape, allowing the end-effector to drop objects quickly. I also did a bit more work on the calibration and the reframing of the image for the model, and added more support to the camera to minimize the need for re-adjusting. I have moved from using a distance sensor to using an air pressure sensor in order to determine when the end-effector is on the object, as the distance sensor has varied measurements depending on the curvature of the object.

I am behind schedule, as there is still some work to do on the pickup sequence.

Posted on

Teddy’s Status Report for 4/19

This week was mainly spent integrating everything together with Ethan and Alejandro. I finalized all of the physical components of the gantry, and created a mount for the camera which sits high above the gantry. I also wrote the code that translates the pixel values of the centroids of the objects into the step distances that the motors need to travel. Due to difficulty getting the software that interfaces with the camera to work, I decided to switch from using the stereo camera to get depth to a time of flight sensor attached to the end-effector. I’ve written some code so that the gantry now stops when the distance sensor is a certain distance away, which is close enough that the end-effector is able to suck the object onto the suction cup.

I plan to finish integration and complete testing/verification next week. I am currently behind schedule.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Strangely enough, I became very familiar with the physical electrical components of the system, such as the stepper motor drivers and the vacuum pump/relay. As ECE majors in CMU, we don’t often have much experience with designing actual circuitry. Luckily, there are lots of tutorials online that explain how these these components work. I learned how to use the vacuum pump and the CNC shield from tutorials on YouTube, since they give a lot of surrounding information that is often cut out of the “How-to” pages on forums or maker websites. 

Posted on

Alejandro’s Status Report for 4/19

This week, I worked on the integration to have the Jetson relay the on-off command to the Arduino, which I accomplished in tandem with Ethan. The webapp can now stop the gantry via the on/off button being pressed. I also rewrote the Arduino code to handle the Jetson commands in a sequential manner. The Arduino can now inform the Jetson when it is done processing a command. Additionally, I fixed an earlier issue I had with parsing the commands of the Jetson so that the Arduino can now receive the coordinates of an object in xy and then move to the corresponding trash bin coordinates.

For next week, I intend to solidify concrete measurements of different components on my end. Additionally, I plan on integrating the web app controls of speed and the Jetson to Arduino interface for that. I also plan on writing the code for updating the statistics on the webapp in terms of how many types of trash objects have been sorted.

I needed to learn how to write Arduino code to control stepper motors. I leveraged YouTube and the webpages of tutorials throughout the internet as I searched for how to do different tasks. I also used the AccelStepper library documentation to help me control the stepper motors. I also needed to learn how to write certain function calls for Node.js js which I learned from googling and reading online forums and such.

I am currently on schedule.

Posted on

Teddy’s Status Report for 4/12

I spent the last two weeks overhauling the entire physical design of the gantry. I replaced the aluminum angles with aluminum extrusions, and used brackets with t-nuts to provide an easily adjustable yet sturdy frame. I made it so that the supporting rod now goes through two wheel attachments which slide across the rails instead of having the rod sit on the edge of the frame. The z-axis rack has been changed from a 3d-printed part to a lazer cut one in order to make sure that it is rigid and straight. I also abandoned the bearing wheels on the z-axis in favor of a 3d-printed guide slot. I replaced some parts of the conveyor belt in order to add tension, and was able to pull the belt taught. I also made a mount for the conveyor belt stepper and added the stepper and the timing belt onto the conveyor, so that the conveyor now moves with the stepper motor. In short, the physical components of the gantry are essentially finished.

Next week, I plan to try to get the depth information from the stereo camera and help work on all of the steps necessary for integrating the work done by my teammates. I am behind schedule but I believe we should get everything done in time.

Verification

For our testing, we will test the gantry’s ability to move with a certain granularity. We’ll move it a certain number of steps and measure its movement in order to determine what the distance corresponding to a step is. We’ll also be doing rigorous testing with multiple common trash/recyclable items such as bottles, cans, cardboard, jars, chip bags, etc. to ensure that the end-effector is able to handle a wide variety of materials and surfaces. We will also make sure to test objects with different weights to determine if it is able to lift objects up to 1 lb. We’ll also be testing the overall accuracy of the depth information (z-coordinate of the object) calculated from the stereo camera.

Posted on

Alejandro’s Status Report for 4/12

This week I mainly focused on getting the path sequencing to work with the motors. I changed the code in Arduino to run on a switch case statement control flow with four main states. There are some bugs with this as when the motors are told to move to a specific trash bin coordinate they move to the origin, which is wrong. Additionally, I implemented the button and the speed control frontend and backend on the web app portion of our system. Now I just need to ensure that the Arduino can take the data relayed from the Jetson and execute it correctly. I also got the web app to receive the bounding box coordinates from the Jetson.

As for the verification of the measurements, I intend to schedule the pick up and drop sequence without suction. My analysis will be based on a scale of time. If the gantry is able to reach a location within 15 seconds and complete the entire execution of pickup and drop off within that time (excluding the suction) then I know that the path sequencing component works as per the design requirements on my end. Additionally, I will test the latency of the web app video feed by running a timestamp on the jetson when it sends data to my server and then timestamp the receipt of it on the server. Then I will perform the analysis by taking the difference of the two times to see if the latency meets the use case requirement.

My plans for the future are to fix the path sequencing code and finalize the canvas drawing on the front end of the web app. I also plan on coding the controls for the conveyor belt. I also need to ensure that the Arduino receives the commands from the Jetson’s relay of commands from the web app interface.

I am currently on schedule.

Posted on

Team Status Report for 3/29

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

One of the most significant risks currently is that the stepper motors will not have enough speed to complete the pickup sequence in 8s. We have had to run the gantry slower than the maximum setting as the frame currently isn’t well-built enough to handle the stresses. We plan to address this by changing the frame to using aluminum extrusions instead of aluminum angles. One other risk is that the stereo camera might not be accurate enough to sense the depth for accurate placement of the end-effector on the trash items. Currently our contingency plan for this is to use a pressure sensor to detect when the end-effector touches the object.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

Other than the change to the frame mentioned above, there are no planned changes to the existing system. This change was necessary as parts of the gantry were bending and buckling during the gantry’s movements. Replacing the aluminum parts costs around $60, which is not much of a problem since after the purchase we still have around $250 of budget remaining.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Posted on

Ethan’s Status Report for 3/29

This week, I worked on setting up the Jetson Orin Nano. I currently was only able to setup PyTorch after a long couple of days because I hit a couple of roadblocks with setting up CUDA and cuDNN, Most of my issues stemmed from incorrect paths that weren’t properly set in the environment variables and incorrect versions downloaded by “pip3 install torch”.  Unfortunately most of progress was wiped as I accidentally did something to the drivers (I have no idea what because I was levels deep in random Nvidia documents). In a better light, I also wrote some code for Monday’s demo.

Currently, I am behind schedule a little. I plan to fix this by continuing to work the Jetson Orin Nano for the rest of this week and early next week.

Posted on

Ethan’s Status Report for 3/22

This week, I started full dataset training for the YOLOv8 OBB model. I implemented the loss function using three tunable hyperparameters that adjust the weight of the bounding box regression loss, the classification loss, and the angle loss. The intent of this was so to show more transparency in the unified loss calculation (the sum of all the previously mentioned loss), and if one loss was too high I could adjust how much the penalty would be. This method would hopefully allow us to better control the model’s convergence and allow for a better check pointing scheme that would save each best model in all four loss types (unified, regression, classification, and angle) to retrain later on. While the model started to train, I started working on creating a Docker image for the Jetson to make porting over the model easier.

Next week, working on some visualization code that would plot the predicted bounding box and class on static images in a clean manner (the core logic of this will be scaled out later for the Jetson)

Currently, I am on schedule.

Posted on

Teddy’s Status Report for 3/22

This week was spent working on getting the vacuum end-effector to work. After some testing and design changes, I was able to get the end-effector to the point where it is able to pick up all 4 types of recycling consistently. I still need to add a release mechanism, which should just be a solenoid which opens and closes a hole in the air flow. I’ve designed and 3d-printed a part that should fit onto the end of the z-axis rack. I’ve also started looking into the stereo camera and the repository that comes with it to get the depth map from the camera. I also spent some time this week ordering parts, including the belt for the conveyor belt, silicone tubing for the vacuum pump, and aluminum extrusions for the gantry. The current construction of the gantry’s frame is hypothetically sufficient, but there seems to be severe wobbling, so switching from aluminum angles to aluminum extrusion should hopefully solve the problem.

Next week, I hope to attach the vacuum pump and suction end-effector to the gantry, get the conveyor belt cloth attached, get the depth map from the camera, and if time allows replace the aluminum angles with aluminum extrusion. I am behind schedule but I am catching up to what was planned.

Posted on

Teddy’s Status Report for 3/15

Most of the time spent this week was on expanding and reinforcing the gantry to compensate for the sag caused by scaling up the design of the 4xidraw. I was able to construct a large frame around the conveyor belt in order to provide a mount for the 4xidraw. I then had to redesign some of the parts of the 4xidraw so that there would be no sag; this involved making one of the rods slide along the frame so that it supported the end-effector. I was also able to print out the rack so that the rack-and-pinion z-axis movement was complete. I’ve also begun testing the vacuum pump with the suction cup. It seems to be able to pick up flat, non-porous objects like plastic consistently, however it does not work as well with porous materials like paper.

Next week I plan to start working on programming the xy movement and will hopefully finish a fleshed out design of the vacuum end-effector. I am slightly behind schedule as I didn’t anticipate the rebuilding of the gantry to take as long as it did.

The picture of the current gantry is shown below: