What did you personally accomplish this week on the project?
Tests are basically finished, most of the metrics were met as described in the final presentation. We are still working on final integration between the gesture recognition UI and Catherine’s AR code. We have actually decided against using the Jetson, as we ran into issues with OpenPose not running/running too slowly on the Jetson. Since we are nearing the final deadline, we have decided it would be easier to do it on a computer instead.
Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
Behind schedule, but we are almost done.
What deliverables do you hope to complete in the next week?
Final product integration.
Unit tests: (This is the same as my old status report)
- To test latency, I’ve simply measured the time it takes for the gesture recognition pipeline to receive the image and then provide pose estimates (essentially model evaluation time). This was simple as inserting “stopwatches” into the codebase, and averaging measurements. The end-to-end latency was about 55 ms, which meets our target.
- For input accuracy, I made “test inputs” for each of the inputs that we wanted to test. Since the inputs are now location based, these tests essentially consisted of holding up my hand over the button I wanted to click on the screen. Overall, this test went very well, since the buttons were fairly large and OpenPose wasn’t noisy enough to “miss” these buttons. Out of the 20 trials I did for a button, 19-20 of them would “pass” (>= 95% acc) — which means registering an input. However, the input accuracy would change under different lighting conditions, meaning that if the room was too bright or too dark, the accuracy would drop to around 18-20 / 20 successes (>= 90% acc), which is still good enough for our purposes.
- For pose estimation, I held my hand relatively stationary, and measured the standard deviation, and max deviation from the mean. For a 1080p camera, this deviation was about 35 px, which is a little more than we’d like (<20px), but is still good enough for our purposes. Note that this metric is bound by the model we choose (in this case OpenPose).
Design changes:
- Switching codebase platform from Jetson to a laptop. There were issues with building some of the required libraries during integration, as well as some runtime issues (OpenPose wouldn’t run). Since we are strapped for time, we switched to having our software system being run on a laptop instead.