Conclusion

In conclusion we were able to combine the parts of our robot system to effectively sort boxes at a rate of 8/min at 95% accuracy. After prototyping a precise and strong robot arm from scratch over 2 years, we used a camera and ML to locate and track QR’s on a moving treadmill then feed this data to a python kinematics engine. We wrote Arduino code to control the arm using motor controllers and angle sensors utilizing position moves and realtime averaged angle feedback.

Here is a video of our system in action
System Video

Here is our Design Presentation Slides
Design Slides

Here is our Final Presentation
Final Presentation

Here is our Final Video
Final Video

Here is our Final Report
Final Report

Marcus Status Report 4/26

This week I worked on dialing the Arduino robot feedback control software. I added servo tuning to only increment by max 10 degree per second, and also update to face downward based on actual position of the earlier links from averaged angle sensor values.

Working with the team to get the robot to have high accuracy at getting boxes every time, and speeding up overall movement.

Team Status Report 4/26

This week we worked on increasing our accuracy numbers by decreasing movement tolerances. We tested each box orientation on multiple iterations, tweaking our timing and move sequencing. We are working to make our overall move/reset time less so we can do more boxes per minute. Overall, we did not make any major design changes this week, and our schedule remains the same.

To minimize this risk, we have moved wires to the inside of the arm and improved the organization of our wiring containers.

Marcus Status Report 4/19

This week I worked on the Arduino feedback control algorithm, and tested robot arm with the Qr detection system. I changed the servo on the 4th axis to have higher volts and refresh at a lower rate, that helped with the jitter and keeping it reliably downward. I also centralized the Arduino, vacuum, motor controllers and horizontal actuator to 3d platform that we connected to a steel plate under the treadmill.

We are working to make our synchronous box pickup more reliable and quick.

Team Status Report for 4/19

This week we continued testing and improving our end to end system (robot arm + vision/control modules). We improved the smoothness and time-efficiency of box pickups and drops via changes to the main control code. We also fixed some bugs with the PC-Arduino connection, coded handling for various edge cases, improved the accuracy of the control logic / commands sent, and finetuned the command sequences to match the new arm changes. We achieved an item sort rate of 7 per minute, such that were able to reliably sort a box and reset completely within 10s.

We plan to run our unit tests again and really finetune those accuracies, as well as continuing end-to-end testing and focusing very tightly on the command sequencing / PC-Arduino interfacing.

Team status report 4/12

These past two weeks, we have been working on getting the robot to pick up a box every time. We integrated the vision/control algos with the robot kinematic algorithms and were able to reliably pick up multiple boxes in a row. Physically, we updated the arm to have both arm and electronic controllers on the same 3d printed structure/piece. We were able to sync the main PC program’s axes/commands almost perfectly with the arm’s angle quirks to maximize accuracy. We were able to test the system on example boxes running on the treadmill and achieve 80% pickup success (8/10 runs). We have already started experimenting with different pickup move paths / command sequencing, as well as experimenting with different refresh rates and power ramping for the servo. Next week we are going to work on dialing in arm movement and test our system with real drop-off bins.

Marcus Status Report 4/12

The last two weeks Ive been testing and rebuilding the robot arm to make it more reliable. I have updated system to have both arm and electronic controllers on the same 3d printed structure, and revised pieces to make wires internal. I tested some pwm ramping functions, and am still revising control software.

I had difficulty threading the wires through the structural components, and adjusting calibration for each new iteration.

Next week we are going to work on dialing in arm movement.

Team Status Report 3/29

This week we were able to initially integrate all of our components (Arduino arm/kinematics code, Python vision/kinematics/control code). We were able to integrate the CV with the robot arm to track qr codes and determine box position/speed/height, send the robot arm sequential commands, and pick up the moving packages. We were able to pick up multiple packages in a row.

Our current biggest risk is that inaccuracies/imperfections in each submodule will add up to make our end-to-end system insufficiently accurate and/or reliable. Currently our vision module is not perfectly accurate, and our arm and kinematics are also slightly imperfect / idiosyncratic. We are ramping up our unit testing to ensure that these issues are minimized. We are also working on optimizing the PC->Arduino command sequencing to minimizing these risks.

Next steps are adding better move selection and qr tracking. We also plan to more definitively test and map out the arm’s exact angle idiosyncracies. Similarly, we plan to do more robust testing of the vision module’s real-world mappings. Best case, by the end of the week, we hope to have the entire box-grabbing system working in a reliable and decently accurate manner.

Marcus status Report 3/29

This week I wrote more code to control the robot arm, and designed a clamp to connect the camera to our treadmill in a robust non moving manner. I isolated the vacuum from the sensors to help with keeping steady voltage feedback. We were able to get the robot to pick up multiple packages in a row. Next steps are adding better move selection and QR tracking.

Team Status Report 3/22

This week we put it all together and got our robot to pick up packages off a moving conveyer belt. This was accomplished by wiring up the robot fully, troubleshooting a lot of angle sensor / actuator issues, and programming the Arduino to run multiple to-position moves. This serves as a big milestone for us as we have achieved a working robot that can complete tasks. We also tested the CV code for getting the QR value, determining the box height, and determining the box speed/position on our complete system with boxes on the conveyor belt. We also tested our timing/speed/latency, and we definitely should have enough time to successfully scan/track/calculate box variables and move the arm into position to pickup boxes before they fall off the belt.

Our next steps are fine tuning our control algos over the coming weeks to pick up multiple packages quickly. We need to dial in our time tolerances and create bins for the robot to put boxes into. We also plan to finish linking our Python and Arduino code, which will enable us to test the QR scanning software and the robotic arm together. Hopefully we can also start testing the entire system next week, since the scanning system can currently acquire all the necessary metrics, and the robotic arm can currently pick up, move, and drop boxes.