Posted on

Teddy’s Status Report for 4/12

I spent the last two weeks overhauling the entire physical design of the gantry. I replaced the aluminum angles with aluminum extrusions, and used brackets with t-nuts to provide an easily adjustable yet sturdy frame. I made it so that the supporting rod now goes through two wheel attachments which slide across the rails instead of having the rod sit on the edge of the frame. The z-axis rack has been changed from a 3d-printed part to a lazer cut one in order to make sure that it is rigid and straight. I also abandoned the bearing wheels on the z-axis in favor of a 3d-printed guide slot. I replaced some parts of the conveyor belt in order to add tension, and was able to pull the belt taught. I also made a mount for the conveyor belt stepper and added the stepper and the timing belt onto the conveyor, so that the conveyor now moves with the stepper motor. In short, the physical components of the gantry are essentially finished.

Next week, I plan to try to get the depth information from the stereo camera and help work on all of the steps necessary for integrating the work done by my teammates. I am behind schedule but I believe we should get everything done in time.

Verification

For our testing, we will test the gantry’s ability to move with a certain granularity. We’ll move it a certain number of steps and measure its movement in order to determine what the distance corresponding to a step is. We’ll also be doing rigorous testing with multiple common trash/recyclable items such as bottles, cans, cardboard, jars, chip bags, etc. to ensure that the end-effector is able to handle a wide variety of materials and surfaces. We will also make sure to test objects with different weights to determine if it is able to lift objects up to 1 lb. We’ll also be testing the overall accuracy of the depth information (z-coordinate of the object) calculated from the stereo camera.

Posted on

Alejandro’s Status Report for 4/12

This week I mainly focused on getting the path sequencing to work with the motors. I changed the code in Arduino to run on a switch case statement control flow with four main states. There are some bugs with this as when the motors are told to move to a specific trash bin coordinate they move to the origin, which is wrong. Additionally, I implemented the button and the speed control frontend and backend on the web app portion of our system. Now I just need to ensure that the Arduino can take the data relayed from the Jetson and execute it correctly. I also got the web app to receive the bounding box coordinates from the Jetson.

As for the verification of the measurements, I intend to schedule the pick up and drop sequence without suction. My analysis will be based on a scale of time. If the gantry is able to reach a location within 15 seconds and complete the entire execution of pickup and drop off within that time (excluding the suction) then I know that the path sequencing component works as per the design requirements on my end. Additionally, I will test the latency of the web app video feed by running a timestamp on the jetson when it sends data to my server and then timestamp the receipt of it on the server. Then I will perform the analysis by taking the difference of the two times to see if the latency meets the use case requirement.

My plans for the future are to fix the path sequencing code and finalize the canvas drawing on the front end of the web app. I also plan on coding the controls for the conveyor belt. I also need to ensure that the Arduino receives the commands from the Jetson’s relay of commands from the web app interface.

I am currently on schedule.

Posted on

Team Status Report for 4/12

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk to our project is accumulated error. Since our sub components depends on necessary communication between each other (machine learning model results to web app and machine learning model results to Arduino and Arduino commands to gantry movement), if one component does not obey their respective design requirement exactly it could risk the whole project’s validation. In order to prevent, we are taking extreme precautions to ensure that the each sub component is working as intended through rigorous testing. Our aim is to mitigate individual risk to mitigate overall risk.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

There are no current changes made to the existing design of the system.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Validation.

We plan to do multiple runs, first isolating by each component of the system (e.g. centroid & classification, depth accuracy, end-effector pickup rate) and doing multiple trials with different types of trash items. We will then do the same with the complete, combined system.

Reliable Sorting:

We will test a variety of trash/recyclables with multiple surface types and materials, in order to make sure that the end-effector is able to pick up objects with 95% success rate. We’ll also measure the distance that the gantry moves over a certain amount of steps in order to determine its granularity in the x,y,z movement directions.

Real-Time Monitoring:

We plan to ensure that the bytes from the Jetson Orin Nano reaches the web app in 30 FPS by timing when they leave the Jetson and arrive to the server using either Wireshark or timestamps in the code of each entity communicating over the network.

Real-time Object Detection:
We plan to use a set of real-life trash objects like plastic bottles and cans. We will do multiple sets (10) of static images each containing a different variety of objects to ensure that the machine learning model can work regardless of the images in the camera frame. We will also need to analyze that labels that the model outputs to see if it lines up with reality. We are aiming to match the 0.70 precision. from the Design Report.

Posted on

Ethan’s Status Report for 4/12

This week, I finished setting up the necessary dependencies for the machine learning model (I was able to download correct PyTorch, OpenCV, and NumPy to be able to use the Jetson Orin Nano’s GPU). Moreover, I met up with Alejandro to start the integration process with the machine learning model and the web app. We decided on a scheme to send messages to the web app. Together we were able to get eh bounding boxes to show up on the web app for real-time monitoring.

Currently, I am behind schedule a little as I need to be met up with Alejandro again to setup a communication protocol between the Jetson Orin Nano and the Arduino.

Machine Learning Model Verification:

In order to verify the machine learning model’s performance, I took pictures of real world trash (empty plastic bottles, soda cans, and etc) on the actual conveyor belt. From there I run the model over the images and plotted the bounding boxes to see how tight of a fit it has on the image. From the image below, we can see that the bounding box is able to cover the object and that gives us confidence that we can hit the ±5 center pixel requirement from our design requirements. Moreover from the timing code I wrote, the model runs is able to run 50-70 ms way below our 150 ms design requirement. And finally, on the validation set we were able to get the 0.70 precision that we almost specified in the design requirements.

Posted on

Teddy’s Status Report for 3/29

This week was spent doing work for the conveyor belt, updated gantry frame, and the end-effector. I was able to construct a rough conveyor belt when the materials arrived, such that the cloth has enough tension where it rolls when the bars it is wrapped around rotate. I spent some time working out how the new gantry frame will be build using the new aluminum extrusions, with the plan being to cut the rods sometime next week for assembly. I was also able to finish the design of the end-effector by attaching a solenoid to the end-effector such that a hole is plugged and unplugged to release the objects grabbed by the suction end-effector. I plan to attach it to the gantry tomorrow so that it is ready in time for demo.

Next week I hope to rebuild the frame and attach a stepper motor to the conveyor belt so that it is able to turn when commanded by the Arduino. I also hope to start the movement estimation code so that the gantry can move to the object’s location given that it is moving at a constant speed after detection. I am on schedule.

Posted on

Team Status Report for 3/29

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

One of the most significant risks currently is that the stepper motors will not have enough speed to complete the pickup sequence in 8s. We have had to run the gantry slower than the maximum setting as the frame currently isn’t well-built enough to handle the stresses. We plan to address this by changing the frame to using aluminum extrusions instead of aluminum angles. One other risk is that the stereo camera might not be accurate enough to sense the depth for accurate placement of the end-effector on the trash items. Currently our contingency plan for this is to use a pressure sensor to detect when the end-effector touches the object.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

Other than the change to the frame mentioned above, there are no planned changes to the existing system. This change was necessary as parts of the gantry were bending and buckling during the gantry’s movements. Replacing the aluminum parts costs around $60, which is not much of a problem since after the purchase we still have around $250 of budget remaining.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Posted on

Alejandro’s Status Report for 3/29

I spent the majority of the week figuring out how to increase the speed on the motors controlled by the Arduino and fixed the vibration issue that would occur with them. The movement produced by the gantry system in the xy plane involved a considerable amount of vibration and slow movement. In order to fix this, I had to modify the C++ code for the Arduino so that the commands sent to the motors were more frequent to keep the motors from stopping at each step that they would execute in terms of their movement (they’re stepper motors). Additionally, I started implementing the code to control the Z axis, in this case the end effector motor. We don’t have a third cable to control the motor so I have to wait for it to arrive in order to verify that it works in conjunction with the other two motors of the gantry system.

My progress is currently on schedule.

For next week, I plan on finishing the code to execute a complete pick up and drop sequence for specific trash types. I also plan on programming the conveyor belt to move once built.  

Posted on

Ethan’s Status Report for 3/29

This week, I worked on setting up the Jetson Orin Nano. I currently was only able to setup PyTorch after a long couple of days because I hit a couple of roadblocks with setting up CUDA and cuDNN, Most of my issues stemmed from incorrect paths that weren’t properly set in the environment variables and incorrect versions downloaded by “pip3 install torch”.  Unfortunately most of progress was wiped as I accidentally did something to the drivers (I have no idea what because I was levels deep in random Nvidia documents). In a better light, I also wrote some code for Monday’s demo.

Currently, I am behind schedule a little. I plan to fix this by continuing to work the Jetson Orin Nano for the rest of this week and early next week.

Posted on

Team Status Report for 3/22

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk to our project is getting the machine learning model to run at a speed that’s close enough to real-time speed.  To do this we plan on making sure that we make a thorough analysis of the speed and accuracy tradeoffs until we reach the most optimal point of performance in the balance between the two. On another note, I next greatest risk is the failure of the gantry system to move fast enough to handle various objects moving along the conveyor belt. A contingency plan is that we’ll have a speed control feature on the web app to slow down the speed in the case that there are too many objects being passed through the conveyor belt for sorting. This would allow our robot to have enough time to sort all of the items being passed through without having to skip any.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

There are no current changes made to the existing design of the system.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Posted on

Alejandro’s Status Report for 3/22

I spent the majority of the week programming the Arduino to control the stepper motors to move the gantry’s xy movement. I spent a significant amount of time figuring out how to move each stepper motor to yield a specific spot on the xy coordinate plane. I can now calculate a specific location on the xy plane and move the location where the end effector is to be placed to that specific location. The C++ code is written inside an Arduino file that controls all of the movement and stepper motors. Additionally, I fixed the issue with the web app streaming by moving the server to run on Powershell so that there are no port confusion or firewall issues when running it from WSL on my Windows machine.

My progress is currently on schedule.

For next week, I plan on writing the code to translate the coordinates given from the Jetson to the values needed for the motors of the gantry due to its unique pulley configuration. Additionally, I’ll work on controlling the end effector once it’s physically been added to the system.