Progress
I started this week by researching what components we need to buy, figuring out what hardware the Roomba SDK and OpenPose supported. I submitted the purchase requests by Tuesday, and hopefully we can get our components ASAP so we can start testing and finalizing our design.
Since our hardware components have not arrived, I spent more time with the team finalizing what gestures we were going to have in the system and discussing what pathing and localization strategies we were going to use for the system.
OpenPose on Google Cloud
In addition, I worked with Rama to begin testing with OpenPose. I got OpenPose to run on Google Cloud GPU (Tesla K80 5000 CUDA cores), getting 0.4s per frame of processing. This is too long for the requirements of our system, but I believe alot of time is in the file IO of writing the sample results onto video. OpenPose also has documentation the website for how to optimize the time of OpenPose. AWS will serve as a backup for our system if OpenPose does not run fast enough on the Xavier board.
Running OpenPose also helped us get a better understanding of what keypoint information to get to identify gestures. We want to start with a heuristic classification approach for gestures, and use a classifier like SVM or neural net if necessary.
OpenPose on Xavier board
When we got the Xavier board, I worked with Rama to get OpenPose running on it. We faced a roadblock since we did not have a computer that fit the requirements from the SDK, so we had to use a VM to flash the L4T OS onto the Xaivier board. We also installed the Jetpack SDK (a collection of NVIDIA CUDA software for ML) onto the board.
We faced a few issues getting OpenPose to run on the board. We had to modify their existing scripts, fix C++ code headers, and add paths to library path to load. At the end, we ended up with multiple Caffe, CUDA, OpenCV versions and an installed OpenPose. However, OpenPose had CUDA errors during runtime, so we are trying to do another clean install.
Schedule
We are on schedule to get OpenPose to run on the board. However, we are more behind schedule on the gesture recognition front since our local computer cannot run OpenPose at a reasonably fast rate.