This week I began some work on the software side. I did some more in depth analysis for what ML models would work well. The final 2 choices were between trt_pose and mediapipe. trt_pose was made by Nvidia and was optimized for our Jetson hardware, making it much quicker. However, it had much worse accuracy compared to mediapipe. By default, the mediapipe installation doesn’t use the GPU on the Jetson, so I worked a bit on building the package from source to optimize performance. I think I got it working, so now the performance is more comparable to trt_pose while having much better accuracy, making it a clear winner. I also did some research into OpenGL for much of our graphics logic as well as some possible compression protocols to improve networking performance.
I think we are still on schedule overall as the software design is pretty mature, so the only big hurdles left for software are implementing and integrating.