This week I am on schedule. This week I resolved a few bugs in the physics model related to an edge case where a ball collision occurs near a wall. This caused the predicted ball location to be out of bounds (beyond the pool table walls). This was resolved using some extra casing on bounds checks. Furthermore I have been working on the ball detections and coming up with potential methods to distinguish between solids and stripes. For solid and stripe detection, I plan to use the colors of the balls in order to classify them. I took some inspiration from the group from a few semesters ago that did a similar project. These tasks are still works in progress which I intend to continue in the coming week.
This week our team also did some testing of our system. Although the system I worked on most was the object detection model, I helped Tjun Jet do some testing on the physics model. The testing we have done this week was rather informal as our goal was to identify edge cases in our physics model. To do this, we ran our system and placed the cue stick in multiple angles and orientations in order to ensure that each orientation and direction would allow for correct predicted trajectories. We wanted to identify any cases that would cause our backend to crash or produce incorrect outcomes. In coming weeks we will come up with more potential edge cases and do some more rigorous testing of the integrated system.
The testing I have done for my individual subsystem is evaluating the accuracy of the ball detections. So far, I have done informal evaluations on the accuracy of the detections in order to determine the ideal lighting conditions required for the model to achieve its best performance in terms of ball detection accuracy. Furthermore, I have evaluated the different cases in which some balls may fail to be detected. To detect these issues, I tried different positions of the balls and observed whether the detection model would perceive them properly (when the detection model runs, it prints out circles around the ball that it detects using cv2’s imshow method). Some cases I tested, for example, were placing several balls side by side; placing balls near the pocket; and placing balls against the pool table’s walls. These tests were done mainly to ensure the proper basic functionality of the detection model.
In the following week, I will conduct tests that are more focused on the accuracy of the ball detections under different possible layouts. Furthermore, I will be testing the accuracy of the pocket locations and the wall detection. For the ball detection tests, the cases I will focus on are:
- When balls are right next to each other (this may make the model have trouble detecting them as circles)
- When balls are right against the walls
- When balls are next to the pocket
- When balls are inside the pocket (ensure they are not detected)
- When the cue ball is missing (ensure that no other ball is perceived to be a cue ball)
In order to test the accuracy of the ball detections in these different conditions, I will be projecting the detections onto the table and taking measurements of the difference between the ball’s centers and the centers of the detections. In our design proposal’s use case requirements, we specified a maximum error of 0.2in.
The procedure for testing the accuracy of the pocket detection will be similar to that of the balls.
In order to test the accuracy of the wall detections, I will also project the detections on the table and measure the distance between the endpoints of each wall to the edge of the actual wall. For this, we are also aiming for a maximum error of 0.2in.
These tests will allow me to evaluate the level of accuracy of our detection models, which will have an impact on the accuracy of our final trajectory predictions.