We started off this week going over what materials we needed to purchase for our project. We compared pros and cons of different product options, looking at their price and their specs. We ordered:
- Roomba 671 – documentation for our Roomba API showed it worked with this model
- Cable to connect roomba to USB – recommended by documentation of Roomba API
- Raspi – To receive data from system and relay commands to the Roomba
- Raspi power supply – To power the Raspi
- Wide angle camera – We found other people using wide angle cameras with OpenPose for gesture recognition.
We built a spreadsheet to track our costs and manage our budget.
We continued the design process by finalizes what gestures our system will handle. We decided on:
- Tele op control with 4 gestures – Tele op commands set with left arm on right elbow
- Forward – Right arm with elbow tilted forward
- Backward – Right arm with elbow tilted back
- Rotate left – Right arm with elbow tilted left
- Rotate right – Right arm with elbow tilted right
- Save home – Right hand up
- Go home – Left hand up
- Drive to pointed location – Right hand point while left hand up
- Drive to user – Both hands up
We also drew out a design for what we want the robot to look like. (with cookies!) This helped us finalize how many more parts we need to buy to build the robot.
We also wanted to experiment with possibly using multiple cameras for our project, with one to identify gestures mounted at desk level and one overhead camera used for localization of user and robot. We want to experiment with OpenPose before finalizing this design. For similar issues, we created an issues tracking doc for project risks. We put up solutions we thought of now, but we want to experiment more with our parts before we can make plans around them.
We started testing with installing OpenPose locally. It ran super slow on our machine (without GPU), 30s per frame. Additionally, we ran on a Google Cloud instance with a NVIDIA Tesla K80 GPU (5000 CUDA cores), which took 0.42s per frame. We don’t need to process every frame our camera gets for our system, but we believe 0.42 is still slow. The example video we ran it on had up to 12 people to run OpenPose on, which could have also increased the latency of it. We hope we can get the runtime down running it on our Xavier board. (500 cores) There are specific methods to install OpenPose on the Xavier board, so we hope we can install an optimized OpenPose that runs faster.
Later in the week, we received our Xavier board, so we have been working to try to run OpenPose on it. The default method for Xavier setup was to use a Ubuntu 16 or 18 machine with 20GB free space, but our computers did not have that much space or the specific linux version. We tried many different ways to install CUDA packages on it, and ended up finding success installing the OS with L4B and using SSH to install jetpack on it. We are still working to get OpenPose to run on the board.