Accomplishments
Over this week, I was able to fully integrate the camera detection pipeline, tracking and kalman filter implementation to make a series of predictions and finally link that to the gantry system. The final result was a system that is able to predict throws from the user on a single axis. I then tried to debug the issue of the 3D depth detection on the Luxonis camera with Gordon, where we eventually deduced that the depth was a no-go for reasons outlined in the team report. We eventually switched gears to using a second camera, which I have prototyped up some code that successfully predicts the horizontal direction of where the ball will land, from a front-facing camera placed behind the gantry system, getting a full-view of the user throwing the ball so that we can use this prediction in-place of the original depth measurement. This uses the same kalman prediction pipeline as the side-view camera and an adjusted detection filter (an addition of MOG background subtraction and also using moments capture rather than contour extraction). Here are the results:
Schedule
Since we have pivoted to a much easier concept by using a second camera, we are back on schedule and implementation details are fully complete – we are waiting for the second tripod to arrive so that we can test the full system. What remains is the balance between the poster/video/final report submissions with the final integration sprint that needs to be completed in time for the final demo.
Deliverables
Testing of the final integrated product will need to be completed once the second tripod has arrived. I will also finish up the code that integrates the side-view camera detection/prediction and the front-view camera detection/prediction whilst waiting for the tripod. I will also finish up some logic that will only allow for the front-view kalman pipeline to be running when the side-view camera has detected the ball in it’s frame, as to prevent the front-view camera erroneously to begin predicting before the user has even thrown the ball.