Weekly Status Reports

This week, I worked on optimizing the physics simulation to improve a simulated shot’s runtime. During the interim demos last week, we noticed that the best shot algorithm would sometimes give a local minimum instead of a global minimum. I touched on this a bit last status report, but this is happening because simulated annealing is inherently random, and the fewer iterations it goes through results in more random outputs. To fix this, I tried implementing the physics simulation in C++, then using pybind11 to call the C++ simulation in the best shot algorithm file (which is in Python). Unfortunately, due to the overhead created by having to convert data from Python types to C++ types, the latency didn’t improve at all — the runtime was much slower because of the computational overhead.

I then tested three different variations to see which had the best runtime: 1) purely Python, 2) Python best shot & C++ shot simulation, and 3) purely C++. I figured that since C++ is an inherently quicker language than Python, option 3 would have the best runtime. However, through testing, I still found that option 1 (purely Python implementation) had the best runtime. I believe this is because Numpy vectorization in Python is incredibly optimized, and the C++ version of Numpy (Xtensor) is not. Below are the average runtimes of the shot algorithm for the three implementations on 10,000 random board positions (note that my computer was not plugged in, so runtime is slightly slower than the 25ms we were getting earlier with option 1).

Purely PythonPython & C++Purely C++

This next week I will look into different ways to optimize the runtime of the purely Python implementation. I am still on schedule and everything I am working on as of now is pretty much post-MVP.

This week, I focused on exploring improvements to the camera system to support motion detection capabilities. A major area of experimentation involved disabling the camera’s auto-blur and auto-focus features. These automatic settings were interfering with consistent image quality, especially during fast motion, so I’ve been testing different configurations to achieve a more stable image feed that’s better suited for detecting movement.

Alongside this, I’ve started preliminary work on motion detection by analyzing frame-to-frame changes between game states. My goal is to develop a reliable method for identifying when and where motion occurs on the table, so that the algorithm can run the physics simulations without user input.

I have also been discussing with my team on weather to pursue the use of a raspberry pi to host our own web server to keep track of previously played games.

I have been experimenting and have made minor progress on these tasks, however, I am on track with my progress.

 

This week, our team made significant progress in refining the camera and projector system, improving the physics simulation, and integrating the core components of our project in preparation for the interim demo.

Luke focused on optimizing the camera’s positioning and settings to improve image capture accuracy. He worked on integrating the camera with the CV algorithm to ensure reliable ball detection and reduce distortion. Additionally, he experimented with different mounting positions for both the camera and projector, addressing stability and visibility concerns. He also adjusted lighting conditions around the table to minimize shadows that interfered with ball detection.

Samuel completed the find_best_shot algorithm, transitioning from a brute-force approach to a more efficient simulated annealing method. This allows the system to find optimal shot angles, power, and spin within a set time constraint. However, due to the randomness of simulated annealing, the algorithm sometimes returns a local minimum rather than the best possible shot. To address this, he has begun rewriting the physics simulation in C++ to improve performance, reducing simulation time from 25ms to under 10ms. While this won’t be ready for the interim demo, he aims to have it completed for the final demo.

Kevin worked on ensuring accurate integration between ball/pocket coordinates and the physics simulation. He also focused on improving error handling for the CV system, particularly in mis-detected balls and pocket positions. Alongside Luke, he helped finalize the table setup, securing the projector mount and ensuring the camera is positioned correctly for accurate detection.

Our team’s next steps will be to finalize the system integration for the interim demo. Luke will continue to resolve the remaining camera system bugs and refining the UI to improve usability and game state visualization. Sammy will continue developing the C++ physics simulation. Kevin will continue to enhance the CV algorithm with ball detection and pocket detection.

This week I was working closely with the team to ensure we had our table prepped for demo which included: testing lighting, ensuring a secure projector mount, and camera angle/distance. Aside from helping the team me and Samuel have been working closely on ensuring an accurate integration between ball/pocket coordinates and physics simulation. Lastly, on the technical side, this week I spent a lot of time on error handling for the CV processing, this included things such as mis-detected balls and pocket position.

This week, I made significant progress on multiple aspects of the project. I continued testing the camera system extensively to ensure accurate and reliable image capture. In particular, I worked on refining the camera’s positioning and settings to improve its ability to capture the game state with minimal distortion. I also focused on integrating the camera system with the CV algorithm, making sure that the image processing pipeline correctly detects and interprets the positions of the balls on the table. To enhance the user experience, I worked on the UI, making it more intuitive for players to capture the game state with minimal effort.

Beyond software improvements, I spent time optimizing the physical setup of the camera and projector. I experimented with different mounting positions and configurations to ensure stability while maximizing visibility. Additionally, I worked on adjusting the lighting conditions around the table to reduce shadows, which were interfering with the CV algorithm’s ability to detect the balls accurately. This involved testing various light placements and intensities to find an optimal setup that enhances detection reliability.

I have been encountering a few bugs with the camera system, but expect to have them resolved before the interim demo. Overall, I am on track with my progress.

^ Image of camera debugging

^ image of table from system

This week, I finished the find_best_shot algorithm and created a variation of the display mapping algorithm that creates the image of the shot we will project on the board.

For the find_best_shot algorithm, I switched the implementation from a brute force approach to a much quicker simulated annealing approach. This approach calculates the initial angle between the cue ball and the target ball (the lowest numbered ball on the board). This angle is either the direct angle between the cue ball and target ball, or the angle that bounces the cue ball off a wall into the target ball. The algorithm then adds a random amount of angle, power, and spin to the previous shot and then simulates it. If this selected shot improves the solution, it is always accepted. Otherwise, the algorithm makes the move anyway with some probability less than one, which exponentially decreases based on the “badness” of the move.

To fit within the use-case requirements, I constrain the algorithm to return the best angle, power, and spin within either 1000 iterations or 4 seconds. This is then passed into the display mapping algorithm to show what the outcome of the shot will be. These images are attached below.

However, as you can see, since the simulated annealing algorithm is essentially random, the 4-second constraint can sometimes lead to different shots depending on what “direction” it takes in hill-climbing. This means that it will sometimes return a local minimum instead of the global minimum shot. For example, in the attached images, the best shot algorithm sometimes returns the shot that just pots the 1-ball (second image), a local minimum, instead of the shot that pots the 9-ball (first image), which is a global minimum since it wins you the game.

To fix this, I have begun to create a C++ implementation of the physics simulation to speed up the simulation time from an average of 25ms to hopefully under 10ms. I will not be finished with this implementation in time for the interim demos, but I hope to be finished with this by the time final demos come around.

I am currently on schedule and will be meeting with Kevin and Luke to integrate our three DeepCue subsections in time for the interim demo.

This week our team continued to make steady and significant progress especially now that we have the physical table and foundation for our project. While there are a couple roadblocks in our physical setup, the software progress is on track, and we are still confident in meeting our desired MVP goals.

Luke focused on setting up the pool table mount for the camera and projector system. He encountered challenges with securely attaching the equipment while upholding safety standards. Initial mounting attempts either made the system unstable or darkened the playing area, impacting out CV system. Luke plans to refine the mount further to balance stability, visibility, and functionality, with the goal of having the camera and projector system fully operational by the end of next week.

Samuel made progress on the find_best_shot algorithm for the physics simulation. His current approach successfully identifies 9-ball shots that meet key criteria, such as hitting the lowest-numbered ball, avoiding scratches, potting balls, and positioning the cue ball strategically. However, the algorithm still relies on a brute-force approach. Samuel is exploring the use of Simulated Annealing to optimize the algorithm and reduce computation time to under four seconds, as required by our use-case. He remains on schedule and aims to complete the algorithm and display mapping in time for MVP testing.

Kevin supported Luke with the physical setup, providing input on camera/projector placement and lighting to minimize image quality issues. He also began implementing code to project predicted holes, pockets, and ball positions onto the physical table. Kevin is on track and will pivot his efforts toward smooth integration, including refining position coordinates and addressing potential image occlusions.

The team is confident in overcoming current challenges and delivering a functional and polished MVP in time for carnival.

This week I worked on helping Luke setup the table and give ideas on how to setup the camera/projector system and lighting in order to minimize image quality issues. Aside from helping with the physical components I started working on implementing code to project my predicted holes, pockets, and ball placements on the physical table. I am on track and will try to pivot my work towards smooth integration, including position coordinates and mitigating image occlusions. Below I have attached an image of the coordinate system we are using and how it looks on the physical table.

This week I spent my time working on the find_best_shot algorithm for the physics simulation portion of our project. Currently, my approach is good and finds the 9-ball shot that: a) hits the lowest numbered ball on the table, b) does not result in a scratch, c) pots 1+ balls, d) sets up the cue ball to be closest to the lowest numbered ball on the table, and e) occasionally pots the 9 ball if possible.

However, I am still using more of a brute-force approach than a greedy one. I am thinking of ways to use some Simulated Annealing to find the optimal shot we can take. This will require tweaking parameters that determine the “score” of a shot — this includes balls potted, where the cue ball ends up, etc. I will be spending my time this upcoming week implementing my algorithm and optimizing it to complete in under four seconds as specified by our use-case requirements.

I am still on schedule and, again, hope to be done with the best shot algorithm + display mapping before carnival so that we can test MVP.

This week, I focused on setting up the pool table mount. I encountered challenges with securely attaching the camera and projector while ensuring high-quality image capture and projection. Some of my initial mounting attempts made the system unstable, posing a risk of falling onto the table. Other iterations created visibility issues, as attaching the camera and projector to the underside of a wooden plank darkened the entire playing area, making it difficult for users to see and play.

I will continue refining the mount to balance stability, visibility, and functionality. My goal is to have the camera and projector system fully operational by the end of the upcoming week.