Thomas’s Status Report for 2/24/24

This week, I presented on behalf of the group for the design review. However, we ended up pivoting away from the pool referee system after the presentation when we got the news that it would be too hard to do with 260 fps cameras. Following this development, I started working on the design requirements for our new project which focuses on tracking a racecar with cameras and auto-generating a livestream from the multiple camera feeds. I wrote around 4 design requirements for each use-case requirement of the new racecar camera livestream project. These design requirements together specify the system at an abstract level and also help us choose a solution approach with the appropriate constraints in mind. One thing I focused in on was requirements for the camera motors, since this will help us in our decision between whether we choose to use servo motors or stepper motors. The motors need to turn smoothly for a good livestream experience, but they also need to be able to start quickly and stop accurately to account for sudden acceleration or deceleration from the racecar.

We are behind schedule since we’ve pivoted to the camera racecar livestream project idea. To catch up to the project schedule, we’ll have to work in parallel on the design document, including the design requirements, solution approach, and testing, while also ordering the parts and doing trade studies to begin the process of actually implementing the system. In the next week, I hope to have trade studies for at least the four major components of our system, leading up to a finalized design document.

Bhavya status report for 02/24/2024

After correspondence with Professor Shamos (who has experience in measuring the game of pool) in which I explained the concept of the double hit detection project, I learned that our project would not be feasible in the way we were planning due to the low shutter speed cameras.

This has put our work behind schedule. But the idea we pivoted too will still have some tracking so not all of my work has gone to waste. I simply have to adapt the pool tracking to car tracking now. I hope to have it done by Wednesday to show off a working model.

I helped flesh out the new idea, select materials and set the use case requirements that we aim to fulfil in this new project.

Team status report for 02/24/2024

After fully defining our project idea for detecting double hits in a game of 8-ball pool, we went ahead to film shots/fouls and get a better understanding of how these shots work. Our research, accompanied by feedback from a professor who has experience in detecting these types of fouls and other pool-based projects, concluded that it would be extremely challenging to detect double hits with only 240fps cameras. Given that our budget does not allow us to purchase cameras with greater shutter speeds,  we have decided to shift our idea. Even though we had some contingency plans to assist our prediction system (accompanying the cameras), the accuracy we were hoping to achieve would simply not be possible.

Instead, we have decided to proceed with an earlier idea of a multi-camera system for tracking a car around a circuit. Given that we cannot test it on a real car, we will be using a slot car track. Cameras placed at various points on the track will pan as they detect and film the slot car. Our system will also stitch together the footage using the feed from the camera that is closest to the car at any given time to output a seamless live feed. The idea comes from trying to automize the filming of trackside cameras in circuit races eg. F1.

Due to the change in plans, we are significantly behind schedule.  We have updated the Gantt chart to adapt to our new idea.  We have ordered the track and redefined all the design presentation goals for our new idea. To get anywhere close to being on track, we are working on having some working model before Wednesday.

To do so we will be working primarily on the tracking algorithm and assessing the racetrack parameters to understand the speed and camera positions we would need to represent real-world scenarios the best we can.

Jae’s Status Report for 2/24/24

This week, our team was able to switch/finalize our idea from pool refereeing to car tracking and generation of stream. Because of this, I was able to put in orders for the things we needed the most at this time, which were the racetrack, arduino, and motors. We also need cameras soon, but for now we are planning on using the camera I bought to test pool double hits. I personally did research on the hardware side, so why arduino uno, why servo motors over stepper motors, and the interfaces between hardwares. I have not started thinking about how to mount the camera stands, but I plan on using some sort of stationary object and attaching a motor onto it, and then attaching the camera onto the rotor. Our group met/talked almost daily to get each part of the design report somewhat thought out, so I personally focused on the system implementation and design studies on the hardware side, as well as testing/verification/validation. We are planning on meeting tomorrow to join these thoughts together and have something to present the TA and faculty. We were unable to make a new schedule yet, so we will do that tomorrow as well.

We are without schedule at the moment, but we know we are quite behind. We hope that finalizing the idea will give us the boost needed to actually start working on the project. We will also be utilizing spring break and some of the slack time to catch up. Personally, to catch up, I will get working on some sort of arduino code to control the motors at fine gradients when the motor/arduino arrive. After that, I will start designing the camera stands.

Next week, I hope to have some form of code ready that allows the arduino to control the camera motor movements given some input from car detection/tracking module. Also I hope to start designing the camera stand, or at least have an idea of what it would look like and how I would build it. Lastly, we will work on the design report all of next week.

Team Status Report for 2/17/24

The most significant risks for the success of the project are hardware limitation concerns and lack of research on the difficulty of the problem we are trying to solve. Hardware limitations could result in project failure because we are buying multiple expensive cameras and if some or all are faulty, we would no longer have the budget to buy more. Further, there is a minimum resolution of the cameras required to achieve the project use-case requirements and design requirements. This risk is being managed by experimentation with whatever cameras we have available to determine a camera resolution requirement, and ordering and returning cameras so we can test them and see if they work. Lack of research on the difficulty of the problem we are trying to solve could result in project failure if the problem is inherently too difficult for us to achieve within our time-frame. This risk is being managed by getting advice from experts who have already done research into the problem. We are currently thinking of contingency plans but haven’t thought of any yet, though we have done initial research towards a pivot to another project idea involving auto-tracking and auto-generation of a stream following a car in case it is necessary.

Since the last status report, we received feedback from Professor Kim and TA Jean that resulted in a change of the use-case requirements, going from a pool foul detection system on most common fouls to a foul detection system on push shots and double hit fouls, two especially hard cases even for humans to identify. This change was motivated by the desire to focus on the most difficult part of the wider problem we were interested in addressing – detection of pool fouls – as the initial wider scope of the problem might have spread us too thin leading to an unfocused project direction. The costs of this change are a schedule setback as we have to begin again from the use case requirements, and these costs will be mitigated by scheduled work sessions in the afternoons. An updated schedule is in progress.

Part A was written by Jae Song.

In general, our product which helps detect double hits and push shots will not meet any specific needs In public health, safety, or welfare. In specific however, we can see how it may play an indirect role in these aspects. In terms of public health, since double hits and push shots are two of the hardest fouls to detect, it may help the players’ psychological health as they may be more at ease to know the line between right and wrong that will be drawn from the product. In terms of safety, the only safety issue our product may raise is the location of the cameras to capture footage to detect fouls, although the danger is minimal threat to the players. Lastly, in terms of welfare, our product emphasizes and highlights fairness, which is what most people consider should be a basic standard in life.

Part B was written by Thomas Li.

Our system addresses the social need of fair competition. In social groups, competition can promote many good things for the group, helping everyone get better and improving quality of life. An autonomous referee provides this need without needing a human to do the job.

Part C was written by Bhavya Jain.

On average new pool tables cost above 1200. Our system is a reasonable addition for any pool lovers who might be looking for a way to officiate their games. The product we will construct may cost around 300-400. If produced on a larger scale, the cost may be reduced. The costliest component of our product will be the multiple high shutter speed cameras. We will be working under the constraints of the budget allocated to us, but if better accuracy is required better cameras may be used with our system which may increase costs. While one odd disadvantage may be pool referees loosing employment, we believe our product would be a tool for them to have more information rather than a replacement product. We hope that the improvements in experience while playing the game might be beneficial to the pool industry.

Jae’s Status Report for 2/17/24

After receiving feedback from the previous week’s proposal, our team took a step backwards to redefine our idea this week. Earlier this week, I communicated with our faculty and TA to clarify their feedback. After validating their feedback and deciding to change ideas, I took charge of fleshing out our backup idea (use-case/challenges/solution/testing) of a camera system that does auto-tracking and auto-generation of stream of a toy car race. I also helped clean up our list of possible fouls to detect for our main idea.

After presenting everything to the staff on Wednesday and part of Thursday, we decided on pursuing the detection of double-hit fouls and push fouls in pool. Then the biggest challenge was determining what the required camera resolution would be to capture a double hit. So I spent some time in the University Center game room performing double-hits and recording them as iphone video(~30fps) or slowmo(~180fps) to see what the images would look like frame by frame. Attached is the frame by frame images (~180fps) of a double-hit.

 

Since there were only 1-2 frames between the two hits, and that the iphone slowmo feature is not quite 200fps, we wanted to see what 260fps footage would look like, so I purchased a 260fps camera. It came today so I will be trying it out tomorrow.

Our schedule has been delayed since our proposal week, so I hope to get back on track by testing out the 260fps camera and: 1. if camera footage is good, team can move forward to rely more on camera footage 2. if camera footage is not quite good, team can look more into using other aspects of the hit to determine if it was a foul or not. I will also be ordering the pool table and cameras this coming week.

Next week, I hope to order the parts needed to start obtaining footage of double hits. I also hope to have a plan to use frame by frame images to detect double hits using collisions as well as distanced traveled by the balls.

Thomas’s Status Report for 2/17/24

This week I drafted a feature list defining the project’s problem scope and proposing initial ideas for the solution. Features were separated into pre-proposal and post-proposal and incorporated feedback from the proposal presentation and were discussed in a follow-up meeting leading to the project direction for the design review presentation. I also drafted a list of the project components that will be part of our solution approach as preparation for the design review presentation, along with a brief overview of what each project component will do and its input and output.

I am behind schedule. To catch up to the project schedule I’ll first need to schedule a meeting with my teammates for us to revise the schedule now that we have pivoted our project away from our proposal idea. Then I can schedule work sessions with one or both them outside of lab sessions to speed up my progress until I am back on track. Next week for deliverables, I hope to complete the design review document. In particular, I should have an explicit interface specified for each of the project’s code files and an overall project code file hierarchy.

Bhavya’s Status Report for 2/17/24

After feedback from our proposal presentation, we pivoted our product from being a partial refereeing system to only a refereeing system for hard-to-detect fouls. This included the double hit and the push shot. There were several brainstorming sessions to figure out how we would tackle this new idea. Given that I am in charge of OpenCV, I still had similar tasks to accomplish. I learned more about the hough circles edge detection method and tested my code on come pool footage to see how I could track the balls. Currently, it is capable of detecting pool balls in each individual frame.  Given that we were only focused on these kinds of fouls, I also went to the UC pool tables to film some footage using different camera angles for double-hit fouls. Since our system depends on tracking fast collisions, I had to know what sort of footage I would be dealing with. I also spoke to other players about their experiences with committing and noticing these kinds of fouls.  On the weekend I helped Thomas with mapping out the solution and implementation process to make the slides for our design presentation.

Given the late pivoting of ideas, we are definitely behind our schedule. However, we are still trying to spend time combing through our design presentation to comb out implementation details.

In the coming week, I will further develop my code to detect pool sticks, add collision testing, and  try to get a lot more footage to test my code given that we will have the pool table and the camera available.

Thomas’s Status Report for 2/10/24

This week, I worked with my team on the project proposal. While brainstorming implementation details for foul detection, I suggested that our team should add a new feature to our 8-ball referee system which allows pool players to “call” the pocket they are aiming for by clicking on one of the pockets of the table in the web app UI. This is necessary to keep track of whether a ball (especially the 8-ball) has been legally pocketed or not according to the rules of the game.

I am behind schedule because I should’ve finished organizing implementation details and edge cases by this point.  According to the schedule, I should begin writing code that relies on camera input next week. To catch up to the project schedule, I’ll do research on Sunday to finish up organizing implementation details and edge cases (finding out what sensor data would be needed if I had ideal sensors). Once I’ve caught up, I can focus on working on next week’s deliverables, which can be pseudocode for the foul detection system that assumes valid camera input.

Team Status Report for 2/10/24

We believe the most significant risk that could jeopardize the success of this project is defining exactly what our system will achieve. What we proposed at first was a refereeing system that will detect and identify most commonly violated fouls in pool. However, after feedback from the professor, we see that depending on the kinds of foul that we are going for, the project will either be too simple and redundant, or very complex and perhaps inaccurate. These risk is being managed by always having an open mind to our project, and be able to adapt to faculties’ advice and continuously defining our idea more and more. Some contingency plans for our idea are narrowing down our fouls to be detected to just one complex foul such as “double hits”. Another plan we have on the back of our minds is a car racing camera system for a toy racetrack, where a system of multiple cameras will track a car through a track and provide a live stream of the camera feed.

No huge changes were made to the existing design. One thing we did consider is removing the cue stick attachments (sensors) and allowing CV to detect stick to ball collisions. This is because it would be hard for sensors to detect these kinds of collisions after looking into it deeper. This change would mean that our computer vision component will be more heavily relied upon and we need more hands on that. Since Jae was to be working on the cue stick hardware, now Jae should be helping Bhavya with any help needed on the CV part or UI part.

Our schedule has been pushed back a bit (mostly for hardware side) since we are still refining our idea and did not want to put in orders beforehand. We hope to put in orders by next week (instead of this past week) to get our schedule back on track.

No photos to report.