PoolTracker DDR

All of the time that I spent on the project this week was working with the group. Because a majority of our accomplishments this week required communication between the different components, this is what made the most sense. I focused mostly on changes with the html and javascript that were required for our changed to the UI.

One of the most important strides was getting the web app to automatically refresh with whatever new data had been sent from the backend. This involved using Ajax to regularly check for updates to graph.

In this coming week, I will be focusing primarily on the work flow of the user with our web app.

As expected, there was limited time to work on the actual project this week due to midterm deadlines as well as the shortened week for Spring Break. Writing the design review report took a lot of time, as we realized there were aspects of the project we still didn’t know we could accomplish.

One of the most important realizations we had is that stroke classification is a much more difficult task than we originally thought. Adithya got OpenPose running on his laptop, but it was unable to perform limb detection on any swimmer. Without limb detection, we can’t even begin to extract information that would be meaningful to a classifier. Because of this challenge, I began searching for alternate ways we might be able to classify strokes without limb detection. The most promising of which is to use k-means clustering to differentiate between the swimmer’s body and other parts of the pool. Upon initial testing, we appear to get the best results with either k=2 or k=3.

I believe that some pre-processing could drastically improve these results, and I’ll be focusing on this in the coming week. I will also be creating the overall UI of our web app since I was supposed to have that completed this week.

The beginning part of this week revolved around the Design Presentation, which was the presentation I was assigned to give for this semester. I spent a couple hours creating/polishing the slides our team made last week, and another hour or so practicing. I feel that the presentation went fairly well and that we got good feedback which revealed weaknesses in our design.

From a technical perspective, I worked with Adithya on furthering progress on our web app. The first task we wanted to complete was to receive data from a remote Python script via POST requests. We ran into a lot of unexpected complications with this, and discovered that we were missing the database infrastructure needed to store what we had received. After troubleshooting this for quite some time, we cut our losses and decided to launch an entirely new EC2 instance, meaning we had to redo most of the work from last week.

Because of this setback, things have gotten pushed back just a little. However, there was a slack week we built in for spring break, so everything is still able to be on schedule.

Given that we have our design report due, I’m attending the reading discussion on Monday, and the ethics assignment is also due, I know that there will be limited time to work on this project. With the few hours that I do have left, I want to get OpenPose up and running on my laptop.

Following our design presentation, we realized that there were some pretty severe issues with our current design. We spoke with a couple TAs and Professor Marios for recommendation of how to move forward, and have made some changes to our approach accordingly.

The biggest risk that we found in our design is with the stroke classification system. Last week, we had gotten rid of OpenPose for limbic detection, but this caused a lot of concern for the course staff. Because of this, we are working in parallel on limbic detection in OpenCV and in OpenPose. Even using OpenPose, there is a large chance that stroke classification will be an intractable task. We still plan to pursue stroke classification as a component of our project, but we must also be prepared to not have completely positive results.

Besides planning to use OpenPose again, we also changed the interface between the backend and our web app such that the backend sends updates at the completion of every length. In our presentation, we said that we would be sending data at the end of every “swim,” but we did not feel that this would be sufficient.

There are no changes to our schedule.

This past week saw a lot of planning and refining of our project ideas. One of the biggest things that I worked on was hashing out the overall architecture of our project, more specifically the what different components of our project would be responsible for and the interfaces between them. I also worked with Adithya to deploy a basic outline of our web app on AWS (pictured below).

We are still on schedule, although our schedule has changed as outlined in our team status update for this week.

For next week, I hope to determine a clear plan for where we’ll get better aerial footage. I will also be working with Adithya to finish swimmer tracking and update the web app if time permits.

The biggest threat to our project are issues with the swimming pool footage. We haven’t been able to find a pool that is willing to help us get perfectly aerial footage of a swim practice/competition, and there isn’t any footage available online that exactly meets our needs. Because of this, we have altered the scope of our project to require the client to manually outline the lanes in the pool that we should be analyzing, and will place additional constraints on the input video as the need arises.

One major change that we made to our project is abandoning OpenPose for stroke classification. We spent two entire class periods trying to get sample code working, but we couldn’t even get the library to compile. OpenPose is designed to be used on GPUs, and without these resources we decided it would be a more substantial project if we did our own limbic detection with OpenCV.

We are still on schedule to complete our project on time, but we have changed the order in which we are implementing things. Because we are still waiting to hear back about collecting better footage, we decided to push back motion detection in exchange for getting a web app up and running. Even though this wasn’t scheduled until the end of March, we have deployed a very basic outline of PoolTrackerDDR on an AWS instance.