Weekly Status Reports

This week, I spent most of my time preparing for my Design Review Presentation. I have also been doing some research on the specific materials we will be using for the frame where we will mount our camera and projector. I have also been weighing the pros and cons of the specific projector our team will be utilizing and narrowed it down to either the Yaber V6, which is better for keystone correction, allowing for +/- 50 degrees of rotation, or the ViewSonic PA503W, which has very good low input lag. I will decide by Monday and have it ordered by Tuesday. I set up some of the Design Review Report already and will be meeting with Samuel and Kyi on Sunday (2/23) to organize and work on the report. The majority of my time in the next two weeks will be focused on the report as well as:

1. Ordering the materials for the frame
2. Building the frame
3. Helping Samuel with simulating slots and assisting with the development of the best shot algorithm

I am a bit behind schedule because I was planning to start the actual building of the frame this past week. However, since the pool table has not arrived, this has delayed my progress a bit. This week, however, I do not have much work from other classes and therefore can allocate a larger portion of my week to building the frame and working on the Design Report and best shot algorithm.

 

This week, I spent most of my time on the Design Review Presentation slides. Additionally, I started working on my portion of the coding for our project. I set up our GitHub codebase and installed/set up the necessary dependencies for the physics simulation. I have also begun to work on our Design Review Report and will be meeting with Kevin and Luke tomorrow and throughout the week to get that done in time. The majority of my time in the next two weeks will be focused on the report as well as coding the following: simulating shots and finding the best possible shot. I am on track based on our schedule, however, next week I will have less time to work on capstone as I have two exams and a project due. Thankfully, I will have ample time during spring break to focus on our capstone project.

This week, our group focused on refining our approach to the computer vision and physics simulation components of the project. After meeting with our professor and TA, we reassessed our hardware requirements and decided that using a single-board computer, such as the Jetson Orin Nano or Raspberry Pi, is unnecessary. Instead, we will run all computer vision and physics simulation tasks on our personal computers, allowing us to prioritize algorithm development while addressing hardware needs as they arise.

Luke spent much of the week assisting with research on both the computer vision and physics simulation aspects of the project. He also worked on the Design Review presentation and Written Design Review, including drafting sketches for the mount that will hold the overhead camera above the pool table. Next week, he will begin constructing the mount and ordering the necessary parts, as the pool table is set to be delivered soon.

Kevin focused on implementing computer vision for boundary edges and pocket detection. He developed edge detection algorithms to determine the table’s physical dimensions and started working on thresholding techniques to identify pocket positions. Additionally, he researched ball detection and categorization methods for later stages of the CV pipeline, compiling useful resources for future implementation.

Samuel concentrated on the physics simulation, researching existing codebases, papers, and videos that could inform our approach. He also conducted initial benchmark testing by playing three games of 9-ball with Luke to establish a baseline for shot efficiency. Furthermore, he updated our block diagram and Gantt chart to reflect the removal of the Nvidia Jetson, as we will now directly connect the camera/LiDAR to a computer.

Overall, our team remains on schedule, and we will continue refining our design while moving into the implementation phase. Next week, we plan to begin assembling the system, conduct additional benchmark testing with the newly ordered pool table, and further develop our physics simulation and computer vision algorithms.

This week, I primarily assisted my teammates with their research on the computer vision and physics simulation components of our project. After our weekly meeting with the professor and TA, we reassessed our hardware requirements and concluded that using a single-board computer, such as the Jetson Orin Nano or Raspberry Pi, is unnecessary. Since all computer vision and physics simulation tasks can be efficiently executed on our personal computers, we decided to prioritize algorithm development and address hardware needs as they arise.

In addition, I dedicated significant time to preparing the Design Review presentation and Written Design Review. This included drafting sketches for the mount that will hold the overhead camera above our pool table.

I am currently on schedule and will continue refining our design while assisting with implementation efforts in the coming weeks. Next week, I plan to order the necessary parts and begin building the mount, as well as order the camera, since the pool table will be delivered by then. My focus will shift toward constructing the system for our project in addition to supporting implementation efforts.

This week, I focused on implementing computer vision for boundary edges and pockets on a full-sized pool table before transitioning to the smaller demo/testing table. The primary goal was to develop edge detection algorithms to accurately determine the physical dimensions and placement of the pool table boundaries in real-world coordinates. This step is crucial for precise ball position calculations during the shot simulation and calculating pocket position. Additionally, I began working on the thresholding portion of the project, which involves identifying pocket position based on the table boundaries and depth. This will help establish a reliable reference corners for ball placement and shot calculation as well.

I also dedicated time to researching ball detection and categorization techniques, which falls in the later stages of the CV pipeline. While most of the week was spent on boundary and pocket detection, I compiled several resources to guide the ball categorization and placement phase. These include GitHub repositories and tutorials on ball tracking and detection:

https://pyimagesearch.com/2015/09/14/ball-tracking-with-opencv/
https://github.com/sgrieve/PoolTable
https://github.com/danilwithonei/billiard_balls_detection

 

This week I did more research into the physics simulation portion of our project. I’ve found some Github codebases + research papers/videos that should be beneficial to our algorithm. I also went to the UC basement to play pool with Luke so we could have a benchmark of how many shots it takes us to finish a game of 9-ball. We played three games and averaged around 40-50 shots. We will use this benchmark as a comparison and will hopefully see a decrease in shots taken when we use DeepCue as help.

Additionally, I took some time to make a block diagram of our project for our design presentation. We’ve made a pretty significant change to our project in that we have decided to remove the Nvidia Jetson and instead directly connect the camera/lidar to a computer. So I updated our block diagram + Gantt chart to reflect these changes. I am still on schedule and plan on working on the physics simulation this week. I will have to coordinate with Luke as he will now be helping with the physics simulation since we aren’t using the Jetson anymore. Additionally, we will conduct more benchmark testing with the smaller pool table that we just ordered.

The majority of our time as a group this week was spent on finalizing and presenting our proposal presentation. Additionally, we have spent some time after presenting to process the feedback we have been given. There aren’t any major changes to the existing diagram of our system, however, there are some things that we are considering now. Luke is looking into tradeoffs between the hardware devices we can use and whether we use them or just have a camera/motion detector connected to a computer. We are also considering changing our MVP to work on the game of 9-ball based on Professor Brumley’s feedback. It would make MVP easier to manage as we would immediately know which ball we have to hit next at any turn.

We have also pretty much settled on the pool table we will be using for our project. It is a 40″ kids-sized pool table (roughly 2/3 the size of a regulation table). We decided on this as it will best fit our budget while still providing us with a working table to use. Additionally, the smaller size will make it easier for the camera to capture the whole board, meaning we can save more budget as we wouldn’t need to buy an extra camera or a more expensive wider lens one.

Ultimately, there is no change to our schedule – we are all on track (if not ahead of schedule) in our respective roles.

Most of my time this week was dedicated to learning how to best figure out the physics simulation for the software side of the project. I have been reading different papers and watching YouTube videos from people to try and best understand how the physics engine should work. Linked below are the papers I read and video I watched. I am also debating between using a graph approach or some sort of heat map with intermediate value theorem for our simulation. Additionally, I looked into Amazon and Facebook Marketplace to try and find any cheap pool tables to use for our project. I am currently on schedule and will begin to start coding the simulation next week. I hope to be able to fully simulate a pool shot within the next two weeks.

 

https://ekiefl.github.io/2020/04/24/pooltool-theory/

https://blog.roboflow.com/pool-table-analytics-object-detection/

https://www.youtube.com/watch?v=vsTTXYxydOE&t=1072s

 

This week, I mainly focused on researching suitable MPUs (Microprocessor Units) for our embedded system, specifically evaluating options that can handle real-time computer vision, physics simulations, and wireless communication. Given our project’s requirements—including 1080p camera input, LIDAR motion detection, and a projector for displaying shot predictions—I analyzed three potential choices:

1. NVIDIA Jetson Orin NX Which Offers GPU acceleration (CUDA, TensorRT, OpenCV) for real-time vision processing and AI-based shot prediction. This would be ideal for handling object detection and physics calculations locally, reducing reliance on external compute resources.

2. Rockchip RK3588 which is a cost-effective alternative with an 8-core CPU and built-in AI acceleration (NPU), making it a good balance between performance and price. It supports multiple peripherals (USB 3.0, HDMI, Wi-Fi) and can offload heavy computations if needed.

3. Raspberry Pi 5 + Coral TPU, a more budget-friendly option that requires an external TPU (Google Coral) for AI-based object detection. This is feasible, but it may not provide the same level of real-time performance as the Jetson Orin NX.

Each option has its trade-offs in terms of cost, ease of development, and computational power. Right now, I’m still weighing the pros and cons of these choices to determine which will best suit our project’s needs.

I am currently on schedule and have gathered enough information to make an informed decision. Next week, I plan to finalize my choice, order the selected MPU, and start setting up the embedded system. My initial goal will be to configure the hardware, and hopefully by the end of the week will have started setting up the embedded system

 

This week, I focused on designing the framework for the computer vision portion of the project. Specifically, I explored image preprocessing techniques to improve input quality for ball detection and categorization, including:

• normalization and resizing image
• converting image to the CIE LAB color space for better color representation
• noise reduction using a filter bank.

To meet computational constraints, I proposed an initial method for ball categorization by using white color detection to separate the cue and striped balls from solids, reducing the reliance on a computationally expensive neural network for this phase. I collaborated closely with Luke Han to align these methods with our embedded hardware’s computational capabilities.

Additionally, I began investigating edge and hole detection for the pool table. For edge detection, I am testing algorithms like Canny and Harris, while for hole detection, I am experimenting with thresholding to isolate shaded regions that represent pockets.

I am on schedule and expect to finalize the image preprocessing pipeline next week, including categorization and detection methods. My next steps will include testing these algorithms on sample table images to evaluate their accuracy and runtime efficiency.