Month: March 2025

This week, our team made significant progress in refining the camera and projector system, improving the physics simulation, and integrating the core components of our project in preparation for the interim demo.

Luke focused on optimizing the camera’s positioning and settings to improve image capture accuracy. He worked on integrating the camera with the CV algorithm to ensure reliable ball detection and reduce distortion. Additionally, he experimented with different mounting positions for both the camera and projector, addressing stability and visibility concerns. He also adjusted lighting conditions around the table to minimize shadows that interfered with ball detection.

Samuel completed the find_best_shot algorithm, transitioning from a brute-force approach to a more efficient simulated annealing method. This allows the system to find optimal shot angles, power, and spin within a set time constraint. However, due to the randomness of simulated annealing, the algorithm sometimes returns a local minimum rather than the best possible shot. To address this, he has begun rewriting the physics simulation in C++ to improve performance, reducing simulation time from 25ms to under 10ms. While this won’t be ready for the interim demo, he aims to have it completed for the final demo.

Kevin worked on ensuring accurate integration between ball/pocket coordinates and the physics simulation. He also focused on improving error handling for the CV system, particularly in mis-detected balls and pocket positions. Alongside Luke, he helped finalize the table setup, securing the projector mount and ensuring the camera is positioned correctly for accurate detection.

Our team’s next steps will be to finalize the system integration for the interim demo. Luke will continue to resolve the remaining camera system bugs and refining the UI to improve usability and game state visualization. Sammy will continue developing the C++ physics simulation. Kevin will continue to enhance the CV algorithm with ball detection and pocket detection.

This week I was working closely with the team to ensure we had our table prepped for demo which included: testing lighting, ensuring a secure projector mount, and camera angle/distance. Aside from helping the team me and Samuel have been working closely on ensuring an accurate integration between ball/pocket coordinates and physics simulation. Lastly, on the technical side, this week I spent a lot of time on error handling for the CV processing, this included things such as mis-detected balls and pocket position.

This week, I made significant progress on multiple aspects of the project. I continued testing the camera system extensively to ensure accurate and reliable image capture. In particular, I worked on refining the camera’s positioning and settings to improve its ability to capture the game state with minimal distortion. I also focused on integrating the camera system with the CV algorithm, making sure that the image processing pipeline correctly detects and interprets the positions of the balls on the table. To enhance the user experience, I worked on the UI, making it more intuitive for players to capture the game state with minimal effort.

Beyond software improvements, I spent time optimizing the physical setup of the camera and projector. I experimented with different mounting positions and configurations to ensure stability while maximizing visibility. Additionally, I worked on adjusting the lighting conditions around the table to reduce shadows, which were interfering with the CV algorithm’s ability to detect the balls accurately. This involved testing various light placements and intensities to find an optimal setup that enhances detection reliability.

I have been encountering a few bugs with the camera system, but expect to have them resolved before the interim demo. Overall, I am on track with my progress.

^ Image of camera debugging

^ image of table from system

This week, I finished the find_best_shot algorithm and created a variation of the display mapping algorithm that creates the image of the shot we will project on the board.

For the find_best_shot algorithm, I switched the implementation from a brute force approach to a much quicker simulated annealing approach. This approach calculates the initial angle between the cue ball and the target ball (the lowest numbered ball on the board). This angle is either the direct angle between the cue ball and target ball, or the angle that bounces the cue ball off a wall into the target ball. The algorithm then adds a random amount of angle, power, and spin to the previous shot and then simulates it. If this selected shot improves the solution, it is always accepted. Otherwise, the algorithm makes the move anyway with some probability less than one, which exponentially decreases based on the “badness” of the move.

To fit within the use-case requirements, I constrain the algorithm to return the best angle, power, and spin within either 1000 iterations or 4 seconds. This is then passed into the display mapping algorithm to show what the outcome of the shot will be. These images are attached below.

However, as you can see, since the simulated annealing algorithm is essentially random, the 4-second constraint can sometimes lead to different shots depending on what “direction” it takes in hill-climbing. This means that it will sometimes return a local minimum instead of the global minimum shot. For example, in the attached images, the best shot algorithm sometimes returns the shot that just pots the 1-ball (second image), a local minimum, instead of the shot that pots the 9-ball (first image), which is a global minimum since it wins you the game.

To fix this, I have begun to create a C++ implementation of the physics simulation to speed up the simulation time from an average of 25ms to hopefully under 10ms. I will not be finished with this implementation in time for the interim demos, but I hope to be finished with this by the time final demos come around.

I am currently on schedule and will be meeting with Kevin and Luke to integrate our three DeepCue subsections in time for the interim demo.

This week our team continued to make steady and significant progress especially now that we have the physical table and foundation for our project. While there are a couple roadblocks in our physical setup, the software progress is on track, and we are still confident in meeting our desired MVP goals.

Luke focused on setting up the pool table mount for the camera and projector system. He encountered challenges with securely attaching the equipment while upholding safety standards. Initial mounting attempts either made the system unstable or darkened the playing area, impacting out CV system. Luke plans to refine the mount further to balance stability, visibility, and functionality, with the goal of having the camera and projector system fully operational by the end of next week.

Samuel made progress on the find_best_shot algorithm for the physics simulation. His current approach successfully identifies 9-ball shots that meet key criteria, such as hitting the lowest-numbered ball, avoiding scratches, potting balls, and positioning the cue ball strategically. However, the algorithm still relies on a brute-force approach. Samuel is exploring the use of Simulated Annealing to optimize the algorithm and reduce computation time to under four seconds, as required by our use-case. He remains on schedule and aims to complete the algorithm and display mapping in time for MVP testing.

Kevin supported Luke with the physical setup, providing input on camera/projector placement and lighting to minimize image quality issues. He also began implementing code to project predicted holes, pockets, and ball positions onto the physical table. Kevin is on track and will pivot his efforts toward smooth integration, including refining position coordinates and addressing potential image occlusions.

The team is confident in overcoming current challenges and delivering a functional and polished MVP in time for carnival.

This week I worked on helping Luke setup the table and give ideas on how to setup the camera/projector system and lighting in order to minimize image quality issues. Aside from helping with the physical components I started working on implementing code to project my predicted holes, pockets, and ball placements on the physical table. I am on track and will try to pivot my work towards smooth integration, including position coordinates and mitigating image occlusions. Below I have attached an image of the coordinate system we are using and how it looks on the physical table.

This week I spent my time working on the find_best_shot algorithm for the physics simulation portion of our project. Currently, my approach is good and finds the 9-ball shot that: a) hits the lowest numbered ball on the table, b) does not result in a scratch, c) pots 1+ balls, d) sets up the cue ball to be closest to the lowest numbered ball on the table, and e) occasionally pots the 9 ball if possible.

However, I am still using more of a brute-force approach than a greedy one. I am thinking of ways to use some Simulated Annealing to find the optimal shot we can take. This will require tweaking parameters that determine the “score” of a shot — this includes balls potted, where the cue ball ends up, etc. I will be spending my time this upcoming week implementing my algorithm and optimizing it to complete in under four seconds as specified by our use-case requirements.

I am still on schedule and, again, hope to be done with the best shot algorithm + display mapping before carnival so that we can test MVP.

This week, I focused on setting up the pool table mount. I encountered challenges with securely attaching the camera and projector while ensuring high-quality image capture and projection. Some of my initial mounting attempts made the system unstable, posing a risk of falling onto the table. Other iterations created visibility issues, as attaching the camera and projector to the underside of a wooden plank darkened the entire playing area, making it difficult for users to see and play.

I will continue refining the mount to balance stability, visibility, and functionality. My goal is to have the camera and projector system fully operational by the end of the upcoming week.

This week, I mainly focused on testing my position coordinates algorithm to validate on the downscaled project table. I also implemented additional fail-safes in my pocket detection algorithm to ensure valid rectangle dimensions based on corner pocket positions, making ball position calculations less error-prone. Additionally, upon receiving the table I had to adjust the algorithm used to compute the homography matrix from the corner stickers because there was an offset from the true corners of the table unlike a traditional layout.

Aside from technical updates, I have been communicating my progress closely with Luke and Samuel who are working on the later stages of the processing pipeline to ensure smooth integration. I have also attached an image demonstrating how position coordinates will be laid out on the physical table and how I am making position calculations less error prone.

This, week our team made tremendous progress in both the physical and software components of our project. We’ve continued developing our major subsystems — computer vision, physics simulation, and the position algorithm.  Luke has made notable advancements on the camera and projector system. And being no longer blocked by the pool table delivery, has assembled, and initial testing has been conducted on the calibration, image capture quality, and integration challenges. However, the frame for the system arrived at the end of the week. This prevented Luke form picking up the frame and completing the setup. He plans to retrieve the frames and complete the setup by Monday, enabling full system integration and further testing. Samuel has completed the physics simulation setup, including the implementation of the simulate_shot function, which accurately simulates pool shots based on angle, power, and spin. He also created the simulate_shot_with_animation function, which visualizes shots and helps fine-tune the parameters for realistic simulation. Samuel will focus on developing the greedy algorithm for shot selection next week, aiming for completion in the next 1-1.5 weeks, in time for MVP integration before the carnival. Kevin worked on testing and validating the position algorithm on the downscaled project table. He also added fail-safes to the homography algorithm, improving the accuracy of the rectangle dimensions and ball position calculations.

Overall, we are making steady progress with our software subsystems and addressing challenges with the physical setup. While Luke’s progress on the camera/projector setup has been slightly delayed due to shipping issues, there are no major changes to the overall schedule. The software side remains on track, with Samuel’s greedy algorithm and Kevin’s integration efforts being the next key focuses.

Next week, Luke will complete the system setup, Samuel will begin implementing the shot selection algorithm, and Kevin will continue refining the algorithms for accurate data processing and integration.