Computer Vision Optimization
We successfully optimized our CV algorithm with our newly purchased hardware: Google Coral USB accelerator. Paired with our borrowed Raspberry Pi 3, our Tensorflow Lite CV library is able to run at 9 to 10 FPS, which is exactly what we were expecting in our initial requirements. If we find this FPS insufficient for future purposes, we will use our remaining budget to purchase a Raspberry Pi 4, which will run at a much higher frame rate.
One issue we found with a bare Raspberry Pi 3 is that it overheats quite easily when running the CV algorithm. We had to wait in between tests on our CV algorithm since our Pi will activate overheat protection and reboot itself. We also experienced a dramatic frame rate drop to 3 FPS when overheated.
Communication with our iRobot Create 2
Since now we’ve successfully optimized our CV algorithm to run at a reasonable FPS, we started connecting the Pi to our robotic base and try to send instructions to it. Thankfully, we found existing library called pycreate2 that will allow us to do it within our CV script in Python3. We were able to send direction instructions, play songs, and receive sensor data that includes multiple cliff sensors and a wall sensor built in. We were also able to make the robot base turn according to user’s relative position to the robot.
The problem with this part is that the CV algorithm is running in a giant while (true) loop, and if we want to let the robot move for a certain period of time, according to previous implementations, the most popular way is to call time.sleep(). But calling such function will affect our CV algorithm and drop our frame rate dramatically. We tried using multi-threading and calling os.fork() to do this but it seems like the Raspberry Pi has issues with multi-threading clients. We will try to solve this problem in the upcoming weeks.
Schedule
According to the schedule, our planned finish time for optimizing our CV algorithm and communicating with our robotic base is intended to be next week. So we are more than one week ahead of our schedule, which makes room for unintended difficulties.