Team C3: Smart Traffic Camera

This week I helped Jon with his new algorithm to isolate moving objects in a frame. It allows us to use a Gaussian Blur kernel to smoothen out the image first, and then subtracts out a background frame to isolate the objects we wish to track.

We know are creating better contours, and are able to better apply the tracking algorithms from last week. From here we are able to create bounding boxes that correctly track the moving object blobs. Once these bounding boxes get to close together, we detect a crash. One thing we are working on implementing is keeping track of direction. For example if two boxes are too close together /intersecting and it’s deemed a crash but they are traveling in parallel, then they should not be a crash as they are probably two cars that looks very close together due to the perspective of the camera but are probably just traveling next to each other in parallel lanes. For example here is an image of two stacked vehicles that show together as one contour. Another issue are shadows, which are also shown in the image below.

This week I worked on further editing the precision of vehicle detection. Our tracking system works pretty well but precision of vehicle detection is still a bit of an issue.

Here are some of the edge cases I worked on:

1. Stacked Vehicles
   

The issue associated with this case is that stacked vehicles prevent the vehicle in the rear from getting picked up until way further when the cars separate.

This is the post back ground subtraction mask. Clearly our system cannot discern that these are two separate vehicles as it mainly uses background subtraction to detect objects on the road.

Solutions I explored for this include utilizing the different colors of the vehicle to temporarily discern that two seperate objects exist.

2) Shadows

Harsh shadows create artificially large bounding boxes as the “background” is being covered by the shadow as seen in this image of the truck. Things become worse in videos captured during hours with “harsh” shadows.

Solutions to harsh shadows include utilizing a “darkened” background subtraction on top of the current background subtraction. This is still a work in progress but essentially we can darken the background and further mask shadows out using the darkened background. Additionally, in this image it is shown that the shadow can be seen “outside of the image of the road.” We are currently working on being able to “mask out” non road pixels when determining contours.

This week we focused on further refining our interim demo based on the feedback we got last week. We presented to Professor Kim virtually last week and are presenting to him in person this up coming week.

Jon and Arvind have been working on making better bounding boxes compared to last week. Last week, we were unable to get super precise bounding boxes that were able to track crashes properly in certain situations, such as with larger vehicles. Therefore, we are working on testing out different bouding box algorithms to see which ones work best for certain situations and how we can determine the algorithm to use automatically. We also want to get the crash detection proven to be working with many different types of video data with different scenarios (E,g. different car types, different lighting etc.) to really show that our method works well.

Goran has continued to work on the rerouting aspects of the project. The main thing he has been working on is to make sure that the models work more generally than our last week’s model that focused on 5th avenue in pittsburgh. We also want to be able to get better simulation of traffic, so that we are able to show how much traffic really got better among certain routes, and how much it perhaps worsened among others. Ideally, we want to generate a table of how the time to get from starting point to destination was updated among different routes after the reroute.

Overall, the project is on schedule and we don’t expect any major changes to our gant chart.

This week we focused on refining our interim demo based on the feedback we got last week. Since Professor Kim was not present in person last week, we will be presenting to him virtually.

Jon and Arvind have been working on improving the bounding box algorithms that are able to track the moving objects in the video frames for crash detection. This algorithm works a lot better than the contours based one we had shown the previous meeting. this algorithm works on removing the background, choosing the background frame from the video feed as one with no (or few) objects other than those always present in the background. After removing this we get outlines of all objects that were not present in the background, and can track them using optical flow packages.

Goran has been working on the rerouting aspects of the project. The rerouting is now able to work interactively with the crash detection aspects. Whenever the rerouting receives a 1 in a text file, it will begin to reroute traffic based on the site and severity of the crash. We are currently using the traffic lights and traffic data on 5th avenue in Pittsburgh as our model.

Overall, the project is on schedule and we don’t expect any major changes to our gant chart.

This week we prepared for our interim demo. I worked mainly on isolating cars from the road for our crash detection system. Essentially the data we were feeding to our system last week had lots of noise present. I worked on further calibrating / post processing the car masks so that we could pass better data to the system.

I used a combination of gaussian blurs w custom kernels and further edited the background removing KNN algorithm (last week I tried using a MOG2 background removing algorithm which worked pretty well but still wasn’t good enough. This week we switched to a KNN algorithm which produced much better results) and in the end we see that much of the noise seen previously has been reduced:

As we can see in the image above, much of the noise from background removal has been removed. Now we essentially use these white “blobs” to create contours and then send that to our object tracking algorithm (different from this and it uses the raw camera data in RGB) to track the cars.

This progress was helpful in gaining more accurate crash detection.

This week I worked on further honing the object detection and tracking schema.

I utilized the MOG2 background subtractor algorithm which dynamically subtracts background layers to mask out moving vehicles. Shown below is a screenshot of this in practice. Using MOG2 works very well for our raw footage because the traffic light camera footage is from a fixed POV and the background is generally not changing. Additionally, We plan on using the masked image below to detect motion and then we plan on using the colored image (as it contains more data) to create the bounding boxes.

I have also made changes to how we detect / track our bounding boxes. This algorithm shown above will detect the vehicles but I am currently working on implementing a CSRT algorithm for tracking  vehicles and using the colored image.