Accomplishments
A lot of progress was made on the kalman filter, including a prototype of an accurate model that is able to track the ball in flight and estimate it’s landing position through visual input. At first, this was done on the YOLO + background subtraction MOG (gaussian) pipeline, however upon integrating this with the OAK-D camera, I realized that this would cap the frames per second (FPS) rate of the camera video output to only 15 FPS, which is much lower than what we need to generate an accurate estimate with the kalman filter. In order to combat this, I decided to pivot to a new object detection model, which is using a colour detection model that detects a specific range of HSV values, also with contour detection for the curves of the ball. This yielded good results for the tracking, which will also help the camera yield a higher framerate as it will not be competing for resources on the camera side. Regarding the kalman filter, I have refined the filter even further to produce good results shown in the video. Although I needed to work on a 3D approach, I spent the time refining the model with the 2D approach before feeling confident enough to work on the kalman filter with 3 dimensions. At the end of the week I also helped Gordon with the initial bring up of setting up the raspberry pi and getting the initial python camera libraries set up on the board.
Schedule and Deliverables
I have made a lot of advances on getting the kalman filter working, so I am on schedule currently. Work needs to be done migrating from a 2D approach to detecting and tracking the ball movement to a 3D approach, so this will be a top priority for next week. At the same time, I will need to find techniques to refine the estimation accuracies of the kalman filter on the 2D version, as it is giving some inaccurate results. I have also observed an oscillation in the predictions that the kalman filter is giving, so I will either have to debug this issue or find a mitigation through a smoothing function or integrating PID control when passing the coordinates onto the gantry system. As Gordon is also working on the bring up of the raspberry pi, some work will be done next week integrating the camera pipelines, installing dependencies, and making sure that the pipeline can run on the board.