This week we were able to initially integrate all of our components (Arduino arm/kinematics code, Python vision/kinematics/control code). We were able to integrate the CV with the robot arm to track qr codes and determine box position/speed/height, send the robot arm sequential commands, and pick up the moving packages. We were able to pick up multiple packages in a row.
Our current biggest risk is that inaccuracies/imperfections in each submodule will add up to make our end-to-end system insufficiently accurate and/or reliable. Currently our vision module is not perfectly accurate, and our arm and kinematics are also slightly imperfect / idiosyncratic. We are ramping up our unit testing to ensure that these issues are minimized. We are also working on optimizing the PC->Arduino command sequencing to minimizing these risks.
Next steps are adding better move selection and qr tracking. We also plan to more definitively test and map out the arm’s exact angle idiosyncracies. Similarly, we plan to do more robust testing of the vision module’s real-world mappings. Best case, by the end of the week, we hope to have the entire box-grabbing system working in a reliable and decently accurate manner.