As the final demo comes closer, we are putting in more and more work to make sure that our project is complete. We have our web app working almost completely, and our tracking functionality working fairly well.

At this point, our biggest risk factor/challenge is going to be getting stroke classification working. We have tried multiple methods for limbic detection with little success. One backup option we have is doing classification with just the raw or processed video stream. While it’s unlikely this will be as effective as classification based on limbic movement, we are hopeful that we can get some positive results.

By the middle of this week, we plan to turn our entire team’s attention towards stroke classification and polishing the tracking functionality.

This week the focus was on getting the web application done to completion. I worked on redesigning some of the UI and writing the logic for the web app backend. The web app can now be used to visualize entire workouts and import/export workouts.

For the upcoming week, the aim is to have our MVP project completed to demo to our TAs and professor on Wednesday. To that end, I will work on getting the stroke classification code completely set up. I will also work on debugging the existing tracking algorithm.

This week I once again spent a lot of my time working with the team in a collaborative hack session. Big strides were made in the web app, including full capabilities added for parsing and handling incoming data. Part of this was adding the ability to read and write a workout to a CSV file, which allows us to develop the web app independently.

I plan to help make sure the web app is completely finished by the beginning of this coming week so my attention can be turned towards improving tracking and joining Karn’s efforts towards stroke classification.

This week, we made some more substantive progress on the actual website. Two of our capstone members have gotten the server to accept real time input from the swimmer, and output a graph of splits on the localhost. We found this task quite challenging, but got it done nonetheless.

The one thing we need to implement is a stroke classifier. In order to do this, we plan to input the relative positions of body parts of a swimmer with respect to the bounding box for “k” frames. The best way to approach this would be to do limb detection, and then input to a NN those floating point values, relative position of body parts with respect to the body parts. Once that has been inputted, we can apply softmax on four output nodes to classify one of four different strokes, and train our model accordingly.

This week I spent a lot of time working with Adithya to put together various components for the demo. We now have it so that the entire process from the backend to the frontend is automated, from detecting a swimmer touching or leaving a wall to their lap time being displayed on the web app.

In particular, we spent a good chunk of time adding filtering capabilities to the web app and also working on debugging the tracking software. While everything does work, the tracker could definitely use some improvement, and I imagine that this is where I will spend a good deal of time in the future.

Next week I am almost completely unavailable as I am building and operating a dark ride for carnival. If anybody is reading this, here’s my shameless plug: Follow @theoldmillride on Instagram to learn more, or stop by Skibo gym 3-9pm any day of carnival for a free ride!

This week most of our time was dedicated to getting prepared for the demo. As a reflection, we think the demo went well and the components that we wanted to showcase for the demo are working as per our initial specification. Based on the TA feedback, we will start working towards improving our front end further and getting stroke classification going.

The immediate next step if to improve our tracking capabilities and setting up a way by which we can start doing stroke classification. We expect the upcoming week and a majority of the remainder of the semester to be spent doing this.

 

This week the majority of the time was spent in fine-tuning our tracking algorithm in order to showcase for our demo. In particular, a realization that we made was that we might need to do our tracking by combining different traditional algorithms that may be suitable to our context. I have been working to get this working smoothly.

For the upcoming week, I’m going to focus on setting up the infrastructure for stroke classification. This involve coming up with a way by which we can analyze our footage, encode the information and feed it into a classifier. This classifier, as was also suggested by a TA, would most likely be an SVM.

This week, we effectively finalized the last steps needed in order to be successful for this project. First, we concluded that it is important to collect tracking data from swimmers after they touched the wall, and display splits on a graph. We have gotten the website to monitor that data, and our time graph refreshes every 5 seconds or so. Adithya is working on the front-end to back-end integration, while Jack is working on the demo, as well as the web-app.

Next, we have to actually do the ML for the stroke classification. This involves “zooming” into the swimmer, and then tracking the moving swimmer. Based on the limbic motions of the swimmer, our software would ideally be able to classify a swimmer on one of four strokes.