Demo 1 and Multi-threading Frame Reads
This week, for our first demo, I showed object detection working on live camera feed at a rate of about 10 FPS and object tracking working on a pre-recorded video of me skateboarding which ran at about 15 FPS. These FPS numbers are concerning, so I looked into it and it turns out that just loading in frames, with no extra processing on the frames, was running at about 30 FPS. The underlying issue here is essentially that video compression and frame decoding takes a longer time than expected. In addition, displaying video makes things even slower — not an issue in our final product but definitely an issue currently during development.
As a result, this week, I’ve been in the process of implementing a multi-threaded version of these frame reads by using a producer-consumer concurrency pattern in which a separate thread loads in these frames into a thread-safe bounded queue so that they can be immediately used by the consumer thread. This is still currently in development but I expect to have this done in the next day or two. After that, I’ll move on to integrating the perception pipeline with the rest of Ike’s work on the CSM and start wrapping up this project!
0 Comments