Category: Team Status Reports

The most significant risk that we need to manage is how late our feed switches. Although we have an algorithm that tries to switch early, our detection picks up the car late, which means our switching is always when the car passes and not before. We are trying to manage this risk by implementing a new algorithm that utilizes a history model of previous camera switching. Then, we are able to remember which camera to switch to, and is able to switch even before the camera detects a car.

This is the change in our design. We are currently in the process of getting it to work, and seeing how stable it will be. The only cost right now is time and perhaps a buggier feed switching, but if it works, we believe it will make our feed switching much better.

We are on track in our schedule. I think we should be testing right now, but I guess we are testing while implementing this new algorithm.

The most significant risk that could jeopardize our project currently is still color detection failures. This wasn’t able to be resolved yet. To mitigate this, we’ve been trying to work in good lighting and selecting our object of interest very carefully to get the correct colors. Another risk that we need to manage is scaling up our system. As we include more cameras, our feed selection algorithm is prone to be more difficult. The increase in overlap of the footages may mean that there are more options for feed selection. To mitigate this, we are currently in progress of incorporating the rate of change of bounding box sizes to our algorithm. We also wish to look into direction findings of the car.

We finally changed our track to the one we will be demoing. We wanted to incorporate a loop and an elevation change. We think this track would be simple, yet very telling of the capability of our system. We plan to use four cameras and locate them as shown on the picture. Other than this, our design is pretty much the same, just scaled up

Our schedule has not changed.

The most significant risks that could jeopardize the success of the project are color detection picking up objects in the background along with inconsistent lighting, and feed selection switching too late or switching randomly for track configurations where there is a slight overlap between camera zones or sharp bends. The first risk is being managed by further research on color detection and asking for help from people experienced in computer vision. The second risk is being managed by considering variations of the track and camera placements to test interesting edge cases of the track before choosing the final track for demo day. Contingency plans are using paint to color the cars in a solid and bright color or placing a lighting fixture or using a room with consistent lighting, and feed selection that takes more information into account instead of being very general.

No changes were made to the system.

Schedule is as shown in the interim demo.

Validation tests will involve running the car on a track/camera configuration test vector for a set amount of laps and reviewing the livestream output for metrics as defined in our user and design requirements, such as latency, smoothness, and optimality of feed choice based on what is in the frame.

Right now, the most significant risk that could jeopardize the success of the project is if our detection algorithm, which is currently done through color detection, is interfered by the environment. Bhavya is working hard on minimizing the interference of the environment after switching over to color detection from object detection(which took too long) but the algorithm is definitely not perfect yet. This is a risk because both the motor control and feed selection modules depend on the accuracy of the bounding boxes outputted by detection. So if detection is a little shaky, the motors could inaccurately pan and feed selection may be wrong. Bhavya is working on mitigating these risks as said in his status report. But the contingency plan is running the system with good lighting and at location/angle where the car along the track never overlaps with another object/background of the same color.

One of the changes we made was using a manual voltage supply to power the race track. We saw that the 9V power supply was providing an electromagnetic field too high, causing the cars to travel fast. By lowering the voltage supply to approximately 4V, the car moves at a slower speed, allowing our system to more easily test. We needed to make this change for now, but we hope to bump up the speed as fast as we can accurately track. No costs were made to our camera system due to this change, since it is a change within the track itself. Another change we made was utilizing simple moving average for feed selection. This means that we are not only changing feeds based on bounding box sizes, but the moving average of it with width of 3. This has been working when the cameras were placed in opposite corners pointing towards the car direction.

Now that we are at the integration period, schedule is up to date.

We didn’t take any convincing photos, but we are looking forward to showing our progress during interim demo next week:)

The most significant risks that could jeopardize the success of the project are the object detection algorithm’s latency and the toy race car’s speed. The object detection algorithm is currently taking around 1 second to product the initial bounding box of the car for the tracking algorithm to use as its input, and this is a problem because we have 4 cameras that each cover a region of the track and they need to be able to get a bounding box on the car from the object detection algorithm anytime the car initially enters the region of the track they are responsible for.  We are working on managing this risk by identifying faster alternatives for the object detection algorithm we are currently using.  The toy race car currently travels at about 1.5 m/s and can go no slower without starting and stopping, and this is a problem because we’re not sure if the cameras will be able to track the car if it goes this fast past the camera, whether because the tracking algorithm can’t keep up, the motor positional feedback control can’t keep up, or the motor itself can’t keep up. We are working on managing this risk by testing out the maximum turning speed of our camera servos and whether that can keep up with a fast moving object at various distances from the track. We are working on a contingency plan for this risk by slowing down the car using weights, and will be testing out taping coins to the top of the car to slow it down.

No changes were made yet to the existing design of the system but the object detection algorithm may change in order to provide a faster initial object detection time in order to meet our system’s latency requirements. This change incurs some extra time working on the object detection before other parts of the project such as auto-tracking and auto-switching can move forward, and these costs will be mitigated moving forward by all team members pitching in to help.

An updated schedule pushes our system integration back a week as we are still working on Arduino interaction and interfacing in CV.

We have gotten our first steps working: object detection of the toy car, tracking of cars in video files, simultaneous control of the 4 servos, and switching between the 4 camera sources. Now we need to improve these parts to meet our project requirements then put these parts together.

As we finalized our design through working on the design doc, the biggest risk of our project is the speed of the car affecting our ability to track it. As of now, we have not done too much on track testing to see if our tracking latency is too slow for a close up camera locations, but our contingency plan is to simply place the cameras further away from the track as needed, to reduce the amount of rotation the camera has to make to track the car.

We made a big switch to change our project to this car tracking one. So in terms of design, we weren’t changing much, just brainstorming for the first time. Through working on the design doc, we were able to come up with challenges we may face, especially in our own parts (hardware, tracking, feed selection). Although we are very late in our idea decision, we are glad we switched as it will be a busy but better experience for all of us.

Our most recently updated schedule is the same as the one submitted on the design document.

Part A was written by Jae Song

On the global scale, our Cameraman product will offer a better approach in live streaming car racing. On the safety side of capturing footage, auto-tracking cameras will hopefully be implemented globally so that no lives are under danger. Car racing is a global sport so our product will definitely have an impact. Not only is global safety being addressed, but we hope that our auto-generation of stream will capture races in a more timely and accurate manner, to enhance the experience of the global audience.

Part B was written by Thomas Li

The multi-camera live-streaming system we design seeks to meet the cultural needs of today’s car sports enthusiasts, specifically F1 racing culture. Primarily, we do this by  providing the viewing experience already established among fans as optimal through algorithmic feed selection. There is already a culture established by race casters, real cameramen, racers, and viewers that we will analyze to determine this optimal viewing experience.

Part C was written by Bhavya Jain

Given our system of cameras closely mirrors that which already exists (with the addition of automated movement, and software for automated feed construction) there seem to be no glaring environmental factors that we feel we must consider. The movement of the cameras will consume some power that was previously a physical force and overall running of the system will consume some power. Motor racing is a very demanding sport in terms of material and power requirements, but our system does not add any significant pressure to these requirements.

The most significant risks for the success of the project are hardware limitation concerns and lack of research on the difficulty of the problem we are trying to solve. Hardware limitations could result in project failure because we are buying multiple expensive cameras and if some or all are faulty, we would no longer have the budget to buy more. Further, there is a minimum resolution of the cameras required to achieve the project use-case requirements and design requirements. This risk is being managed by experimentation with whatever cameras we have available to determine a camera resolution requirement, and ordering and returning cameras so we can test them and see if they work. Lack of research on the difficulty of the problem we are trying to solve could result in project failure if the problem is inherently too difficult for us to achieve within our time-frame. This risk is being managed by getting advice from experts who have already done research into the problem. We are currently thinking of contingency plans but haven’t thought of any yet, though we have done initial research towards a pivot to another project idea involving auto-tracking and auto-generation of a stream following a car in case it is necessary.

Since the last status report, we received feedback from Professor Kim and TA Jean that resulted in a change of the use-case requirements, going from a pool foul detection system on most common fouls to a foul detection system on push shots and double hit fouls, two especially hard cases even for humans to identify. This change was motivated by the desire to focus on the most difficult part of the wider problem we were interested in addressing – detection of pool fouls – as the initial wider scope of the problem might have spread us too thin leading to an unfocused project direction. The costs of this change are a schedule setback as we have to begin again from the use case requirements, and these costs will be mitigated by scheduled work sessions in the afternoons. An updated schedule is in progress.

Part A was written by Jae Song.

In general, our product which helps detect double hits and push shots will not meet any specific needs In public health, safety, or welfare. In specific however, we can see how it may play an indirect role in these aspects. In terms of public health, since double hits and push shots are two of the hardest fouls to detect, it may help the players’ psychological health as they may be more at ease to know the line between right and wrong that will be drawn from the product. In terms of safety, the only safety issue our product may raise is the location of the cameras to capture footage to detect fouls, although the danger is minimal threat to the players. Lastly, in terms of welfare, our product emphasizes and highlights fairness, which is what most people consider should be a basic standard in life.

Part B was written by Thomas Li.

Our system addresses the social need of fair competition. In social groups, competition can promote many good things for the group, helping everyone get better and improving quality of life. An autonomous referee provides this need without needing a human to do the job.

Part C was written by Bhavya Jain.

On average new pool tables cost above 1200. Our system is a reasonable addition for any pool lovers who might be looking for a way to officiate their games. The product we will construct may cost around 300-400. If produced on a larger scale, the cost may be reduced. The costliest component of our product will be the multiple high shutter speed cameras. We will be working under the constraints of the budget allocated to us, but if better accuracy is required better cameras may be used with our system which may increase costs. While one odd disadvantage may be pool referees loosing employment, we believe our product would be a tool for them to have more information rather than a replacement product. We hope that the improvements in experience while playing the game might be beneficial to the pool industry.

We believe the most significant risk that could jeopardize the success of this project is defining exactly what our system will achieve. What we proposed at first was a refereeing system that will detect and identify most commonly violated fouls in pool. However, after feedback from the professor, we see that depending on the kinds of foul that we are going for, the project will either be too simple and redundant, or very complex and perhaps inaccurate. These risk is being managed by always having an open mind to our project, and be able to adapt to faculties’ advice and continuously defining our idea more and more. Some contingency plans for our idea are narrowing down our fouls to be detected to just one complex foul such as “double hits”. Another plan we have on the back of our minds is a car racing camera system for a toy racetrack, where a system of multiple cameras will track a car through a track and provide a live stream of the camera feed.

No huge changes were made to the existing design. One thing we did consider is removing the cue stick attachments (sensors) and allowing CV to detect stick to ball collisions. This is because it would be hard for sensors to detect these kinds of collisions after looking into it deeper. This change would mean that our computer vision component will be more heavily relied upon and we need more hands on that. Since Jae was to be working on the cue stick hardware, now Jae should be helping Bhavya with any help needed on the CV part or UI part.

Our schedule has been pushed back a bit (mostly for hardware side) since we are still refining our idea and did not want to put in orders beforehand. We hope to put in orders by next week (instead of this past week) to get our schedule back on track.

No photos to report.