Month: March 2019

Team

With the midpoint demo next week, this week was focused on getting our final elements that we want to show finished. We worked on the correction algorithm for stills, which will be run through a script for the demo, and on creating a UI to show what our final project will look like. In the upcoming weeks, we will work on fully connecting the front and back ends for a good user experience and a working project.

Kristina

Since this week was so busy for me, most of my work  will be front-loaded into the upcoming week to get it done by our actual midpoint demo. I’m working on creating a UI skeleton and a starting portion of the connection between the front end and back end. After the demo, I will start fully connecting the application together and integrating the text to speech element.

Brian

This week I worked on completing the necessary items for the initial demo. I was able to create a foundation for the pipeline that is able to take an image and output a correction within a couple of seconds. You can customize what moves you are trying to work on, as well as how many corrections you would like to receive at a time. The program will spit out the top things you need to work on in a text format. It will also draw a diagram of what your pose looked like, with circles over the areas that you need to correct. For next week, I would like to start working on corrections for videos, and work with the movement data that we will be collecting soon.

Umang

This week I worked with Brian to complete the demo; particularly, I worked on an end to end script that would run our demo from the command line, take a user captured image, give the necessary corrections to deviate back to the mean ground truth example. Based on the command entered, the user can denote which dance move they would like (and which version of the pretrained  model — fast or not). Next week, I will be traveling to London for my final grad school visit, but I will be thinking about the linear interpolation of how we will frame match for videos of dance moves. I also hope to leverage the increased training data to run the current pipeline with more fidelity.

Team

This week, our focus is all on the midpoint demo. We spent some time this week deciding what we want to show as progress. After some discussion, we’ve decided to focus on the correction aspect of the project as opposed to the user experience and interaction with the application. We have an accurate joint estimation that we’re using to get the coordinates of the points, and have gotten line segments from that point, so we’ll have to focus on getting angles and correcting those angles in the upcoming weeks. The three of us unfortunately all have especially busy schedules in the upcoming weeks, so we are also making sure to schedule our working time so that we don’t get behind on the project.

Kristina

My main focus this week was gathering the data needed to establish the ground truth. We’ve decided that we want to gather data from multiple people, not just me, for testing purposes, so I’ll continue meeting with some other dance (and non-dance) friends to collect data into the beginning of next week. I will also help in testing our processing speed on a CPU vs a dedicated GPU to see if we should buy a GPU or update our application workflow. This upcoming week will probably be one of the busiest, if not the busiest, weeks of the semester for me, so I will focus on work for the demo and will continue work for my other portions of the project afterwards.

Brian

This week I focused on creating all of the functions necessary to process the data and extract the necessary information from it. I was able to create the general foundation that is able to take the images, extract the poses from them, and collect the angle distributions. I have also started creating our example pose collections for use in comparing with the user data. By next week, we would like to a having a working demo for still correction for 3 moves that is able to serve as a proof of concept for the following work on videos.

Umang

This week I focused on building out our core pipeline. I am able to convert an image (or a frame from a video) into a pose estimate using AlphaPose. Using those poses, I worked with Brian to calculate the angles between the limbs found on a given pose (as per our design document). Once Kristina collects the requisite data (stills of multiple people doing the same pose), we can get a ground truth distribution of the true form for three poses. By the midpoint demo day (4/1), we hope to extend the aforementioned to include the variance ranking, which would tell us which angle to correct. Thereafter, we hope to check whether we should use a GPU for pose estimation and we hope to develop our frame matching logic for video streams.

Team

At the beginning of the week, we focused a lot on finalizing our design and completing the design document. After that was done, we worked on our individual portions of the project. Writing the design document took a lot more time than originally estimated, however, so we didn’t end up spending as much time on actual implementation of the project as we had previously hoped. Though since we didn’t make any big changes again this week and we had given some time for the design document, we believe that our work is still on track towards the midpoint demos.

Kristina

In addition to spending a lot of time working on the design document with Brian and Umang and working on other capstone assignments due (Ethics assignment), I started collecting the ground truth data. The first step in that is creating an application where I can perform a pose multiple times in front of my camera and the joint data will be saved in JSON format. Once I’m done creating that application, I will collect multiple instances of every pose that we are aiming to do. My goal is to have that complete in the next couple weeks so that we have data to test soon after we get back from Spring Break.

Brian

This week was spent working on the algorithms to detect the difference between the user poses and the instructor poses. I took some example json data to help with the task. I will continue working on this after Spring Break, and hope to finish that week on initial construction of the algorithm. I also worked on the ethics assignment, as well as further refinement of our design.

Umang

This week I worked on the angular domain calculations. How can we find a scale and shift invariant domain wherein we can compare pose estimates. Due to PhD visits during spring break, I won’t be able to contribute to this over the coming week. However, I hope to finish the transformation specifics by the end of the following week such that we have a rough draft of our pipeline by the first week in April.

Team

This week was focused on the completion of the design, and thinking through many of the important aspects of our design. We didn’t make any sweeping changes that affect the course of the project, and are still on track. We just need to start implementing some of the planned ideas this week.

Kristina

This week, I worked with Brian and Umang on refining our design presentation as well as our design report. I didn’t work a ton on the actual implementation of the project, but helped with many design decisions as we finalize our design report. This upcoming week, I hope to finish gathering the expert data needed for the project.

Brian

Most of this week was spent on the design specifications for our project. We still had a couple of implementation details to think through, so that was the main concern. A big realization was that we need to transform the data into the angle space rather than look at the points themselves. This will allow us to account for the scaling and translation of the person within the image easily. Next week I would like to have an implementation for a still image running so we can move on to movement afterwards.

Umang

This week I had to travel for grad school visit days; nonetheless, I contributed to the design document development. Moreover, during a collaboration session with Brian, we realized that we need to map the limbs from Euclidean space to the angular domain. As such our feature vector would be six angles at the joints of a person. Using a rule based system, we can map each angle to a movement (given two angle feature vectors from adjacent frames) and then prompt the user with a directional correction phrase (after the Mozilla TTS). Next week, I hope to have built the angle feature vectors.