Author: ishaang

These past 2 weeks, we have finally completed Ballbot and have tested different parts of our project. I helped redesign the casing for the hardware , added a GUI for the touch screen, and also tuned the computer vision to work with outdoor tennis courts. Instead of just having the hardware on the roof of the robot, I helped design a second layer to go on top of the hardware that would also encase the touchscreen. We decided to add a touch screen to the top of the Ballbot so that it is easy for a user to turn the Ballbot on and off.

For the UI, I made a simple GUI in Python with 3 buttons to start and stop collecting balls, as well as turn Ballbot off. I integrated the GUI with our existing software so that controlling Ballbot is as easy as possible. To do this, I used Tkinter for the UI elements and the Threading library to have a thread waiting for user input while the rest of the code is running. On the software side, I also tuned the computer vision thresholds to better work with out door lighting. This allowed us to test the Ballbot outside and see it work on the tennis courts.

This week we made progress towards finishing the physical aspects of the robot and placing the finishing touches in the software. We finished constructing the basket to carry the tennis balls. We ensured that the basket to fill 20 to 30 balls so that we could meet the requirements we laid out. The basket was made with waterproof plastic to make sure the basket was lightweight and waterproof. We also used the same swivel wheels as used on the arms of the robot to lift the basket off the ground. We still have to finalize how to attach the basket so that it is secure.

On the software side, we added functionality so that when no balls are detected, the robot spins in a 360-degree circle to see if there are any balls outside of its view. If so, it stops spinning and goes after those balls. If it still does not find any tennis balls, the robot stops spinning and waits until a ball is seen.

This week, we were not able to test on the tennis courts as panned because of rain. However, the weather app says that there will not be rain on Monday, so we hope to test on the tennis courts then. In the meantime, we have tested indoors and everything seems to be working as planned.

This week, I worked on integrating the software and mechanical parts of the project and testing them together. The software required the Nvidia Jetson Nano, Intel Camera, and buck converter to give the Jetson Nano 5V of power which were not part of the mechanical tennis ball launching system. We made an acrylic roof for the Ballbot on which we placed all the hardware required to run the software as well as the battery and motor controller which were part of the mechanical system. In order to connect the battery to both the motor controller and the buck converter for the Jetson Nano, we soldered wires onto a protoboard that we also placed with all the other hardware. To secure the hardware to the acrylic roof, we used M3 dual lock adhesive which acts similar to Velcro.

Once the components were all on Ballbot, we first adjusted the speeds of the motors so that we were using minimal power while still ensuring balls would not get stuck partway up the ramp. Then, we finally started running our computer vision code and tested everything together.

Our integration worked successfully and the Ballbot was able to go after tennis balls and launch them towards the back of the robot. Next week, we plan on adding the basket to the back of the robot so that the balls are collected in the basket instead of just being launched back.

Also, we painted the exterior of the wood so that the arms Ballbot would be weatherproof. We still need to enclose the hardware to ensure the hardware can survive in the outdoors. Also, next week I plan on adding code to the software so that if the Balbot does not see a tennis ball, it turns in a circle once and ensures there are not any tennis balls outside of its field of view. Currently the Ballbot just stops when it does not see a ball which is not the best for picking up all the balls. We also hope to test the Ballbot on a tennis court if there is good weather next week.

This past week, I made progress on making the Jetson Nano and all the other connected hardware portable with the iRobot Create 2. To do this, I first figured out how to power the Jetson Nano and the Intel depth camera with the rechargeable battery we had. After testing that all our software worked as normal on the Nano when it was powered by the battery, I moved to testing the software with the Create 2 robot free to move after the tennis balls. This was not possible earlier since the camera and Jetson Nano were being powered from the wall outlet. We temporarily taped the camera to the front of the robot and also placed the Jetson Nano on top. When Rashmi held a tennis ball in front of the robot, the robot accurately followed the tennis ball, which shows that our software is working.

On the hardware side, I helped Ryan attach the arms and motors to the Create 2 robot so that the tennis ball collection system was mostly complete. We screwed the acrylic ramp to the arms and tested the hardware system. We manually controlled the robot and were able to launch the tennis balls that were funneled into the robot.

Next week, I plan on working on integrating the software and hardware into 1 system and also building the platform on which we will place the Jetson Nano, Intel camera, and battery. The 2 systems seem to be working well independently, but merging them together will be challenging.

This past week, I continued to work on the software to get it to build on the Nvidia Jetson  Nano and also helped on the building side of things.  For the software, I was finally able to get the iRobot Create 2, the Intel Realsense Camera, and the OpenCV computer vision code all running together on the Nvidia Jeston Nano. This involved finding a workaround to building the Intel Realsense Python library from source as well as fixing some other issues related to transferring the code from a Windows implementation that was running on my laptop to a Linux now for the Jetson Nano.  Now, when running the code, the Create 2 robot turns towards the closest tennis ball in view of the camera until the tennis ball is centered. To better test this, we need to power the Jetson Nano from a battery instead of the wall port. This will allow the entire system to be portable instead of tethered to the wall.

On the hardware side, I helped Ryan construct some of the wood parts of the Tennis ball launching system. More specifically, we worked on building the sides of the ramp that the tennis ball will go over, as well as the pieces that the motors are connected to. To connect the pieces of wood, we decided to use wood glue instead of screws so that we were free to cut however we needed.

Next week, I plan on trying to power the Jetson Nano with a battery and seeing if there are any issues with that. Also, I will try to get most of the code written to control the motors. I will help Ryan continue working on the building side of things so that hopefully the motors are ready to be tested along with the rest of the hardware.

This week I worked a lot with the computer vision algorithm trying to make it more robust. I took the camera outside and tested the algorithm in daylight. At first, even if the camera angle changed a little, the ball stopped being detected. Furthermore, I was outside for a few hours and I noticed that as the daylight itself changed, I found that I had to retune the algorithm to work. However, after rewriting some of the code and tuning the parameters of the HSV color space, I believe that I have finally found a happy medium threshold where the tennis ball is detected all the time in daylight.

I also spent some time this week helping my team write the design review document. We were hoping to have feedback from the professors regarding our design process by now but we have not heard anything.

Next week, I plan on working with Rashmi in order to integrate her working with regards to the motion planning of the iRobot and my computer vision algorithm. I’m not sure how this process is going to go as neither one of us has really worked in the other’s domain. I anticipate that this process of integrating it and then finetuning the motions and fixing the issues that arise will take us this whole week and possibly some of next week.