This week I mainly worked on incorporating different nodes of the python DepthAI package to work with our new algorithm for the depth camera.
Our depth camera now incorporates a feature tracker that finds and tracks image features from RGB camera frames, and I successfully incorporated the RGB frame with depth frames, so now for each feature generated by the feature tracker, we can know its position in the frame and the distance data in the corresponding depth frame (available for comparison with the baseline if necessary) across different frames.
I’ve also tested the performance of the feature tracker with Kelton, and we found that the feature tracker performs poorly when the camera is shaken, in which case we may lose track of the features. We are considering the necessity of adding an IMU node in our depth camera code, but more testing is required to decide.