Accomplishments
Over the past two weeks, I was able to complete the 3D implementation of the kalman filter – this is good news as this means all the coding on my ball detection, tracking and prediction has been completed! I have also completed the integration of the camera pipeline onto the raspberry pi, which means that it is able to feed coordinates to the x-y gantry system. Some obstacles that I ran into when integrating the pipeline included the frame rate problem – when taking camera frames in the main while loop, the detection and kalman code would take a non-trivial amount of time. This would cause the framerate of the overall pipeline to drop to about 15 FPS, which is not enough for our needs. The solution we discussed was to implement multithreading – one thread to take in camera frames and run the detection filter, and pushing the detected ball in x-y coordinate form into a synchronized queue data structure. The second thread would pop coordinates out of the queue, and run the kalman filter to output the predicted coordinates, which would be fed into the serializer module for the arduino. This solution deemed to be extremely successful, allowing the framerate to increase to 60FPS, which exceeded our expectations.
Schedule
Currently, we are a bit behind schedule as we still need to fully test and verify our system is correct. However, we are opting to use the slack time that we allocated ourselves for this task, and so we are confident that we will be able to complete all these tasks within the final demo day.
Deliverables
I need to finish testing of the 3D kalman filter with the real-time data once Gordon has fully tuned the depth information from the camera and also once the white backdrop has arrived (so that we can fully detect the ball in all lighting conditions). I will also need to tune the parameter of the kalman filter, so the gravity parameter in real time calculations is different to the gravity parameter used in the initial testing videos. I will also help Josiah and Gordon to make sure that whatever needs to be done is completed before our final demo.