Posted on

Team Status Report for 4/24/2021

A risk to our project is not being able to run the UI on the board. It could possibly run differently on it. Another risk is the pose comparison being too slow or the UI lagging once we get both on the board. To mitigate any problems from integrating all of our software on the board, we will get it on the board by Monday so we can start testing how everything works together.

One change was made to our materials. We bought a second camera (same model as the one we already have) because we ran into issues with recreating bugs (we encountered on the Xavier) locally on Zixuan’s laptop. We found that the difference in frame rates of the laptop camera and web cam on the Xavier was affecting the behavior of the pose comparison code. This only increased our project budget by $30.50.

Posted on

Sarah’s Status Report for 4/24/2021

This week, I added the evaluation workout to the UI loop. Now, it comes directly after the instructions screen. Once the user finishes the evaluation workout and is show their score, it’ll take them to a workout instructions screen (to inform them that the next workout will be a personalized based on the score they received on the evaluation workout). Then, they can choose when to start that workout.

I also made the rest screen look the same as the exercise gif screens. So, now all of the screens have the demo gif on the left and the user live video feed on the right. The label of the current exercise is on the top left corner and the label of the next exercise is on the top right corner. On the rest screen, there is also a label in between the current and next exercise labels that displays the rest countdown.

Lastly, I created a help screen that lists the quit, return-to-home, and pause commands. This help screen can be accessed from anytime in the UI except for the actual workout. So, the user can press “H” on their keyboard for help if they are on the home screen, any instructions screens, exercise preview screens, and scoring screens. I only allow the user to press quit the entire UI program, return to the home screen, and pause their workout once the workout is running. If the user isn’t doing their workout and goes to the help screen, they can press enter to return to where they left off.

I’m on schedule (we’re currently in our buffer time).

I’m planning on uploading the UI code to the Xavier board so I can integrate it with the pose comparison for scoring. I also am planning to add more exercises to our exercise library so we’ll have at least 100 exercises.

Posted on

Team Status Report for 4/10/2021

The most significant risk to the success of our project is the TX2 board freezing or failing to execute our pose comparison code. It used to run once in 6 tries, but now it runs half the time. The failures are caused by memory errors. We think it might be using too much memory because we keep getting memory notifications. If we restart the board, we found that the memory errors are less likely to happen. But we want to find a more permanent fix.

We do have a backup plan of running everything on a laptop, but we don’t think we’ll need to resort to this backup plan.

 

Posted on

Sarah’s Status Report for 4/10/2021

This week, I added labels to the exercise and rest screens. The labels are of the current exercise name (or “rest” if rest) and the next exercise name. I also made the demo video slightly larger to better match the camera feed’s size on the screens. I found a stretch video for the end of each workout, I just need to add it to the UI. I will be playing only a portion of it. I also started creating a workout preview screen before the actual workout starts (so users can familiarize themselves with the exercises in the workout).

This week, I also met with Maddie (on Sunday, Monday, Wednesday) to help set up the TX2 and download the necessary libraries/code onto it. For some reason, the installation of the NVIDIA package froze around 40% for her, and when we tried it on a lab computer, the virtual box couldn’t connect to the CMU wifi so we weren’t able to start the installation of the NVIDIA packages for the board. We eventually got the installation to work when I installed the virtual box on my laptop. I tried to take the board home and finish the installation there, but the board couldn’t find my wifi, so I finished it the next day with Maddie.

I am on schedule.

I hope to finish the workout preview before the interim demo.

Posted on

Zixuan’s Status Report for 4/10/2021

This week I finished the code for pose comparison. We can enter the exercise name and number of loops we want to do, and the program will return the angle differences for each loop, and the sum of angle differences after the exercise is done. Later in the week I also modified and finished the code for frame selection, which looks for the frame with best fit pose for each frame we are comparing. Thus the users are allowed to do the exercise at slightly different speeds. For example, if we are comparing frame 10 of an exercise, we will look at frame 5~15 of the user video, and find the frame with the most similar pose to the standard one. I have tested it several times by doing some of the exercises myself, and the output does reflect my performance.

We were able to run OpenPose on TX2 successfully, however, there were some issues related to memory allocation when running my code on TX2. The error would occur about half the time we run it, and we want to fix that soon.

We are on schedule. Next week I will do more tests with the pose comparison code and fix some bugs.

Posted on

Maddie’s Status Report for 4/10/2021

This week I spent a lot of time using the TX2 and transferring our code from CPU to GPU.

Me and Sarah were able to finish all of the Jetpack and CUDA installs using a Virtual Box on her laptop, since I came to the realization that my USB-C to USB converter was being finicky and my computer wasn’t recognizing that the TX2 was plugged in.  I also spent a lot of time in the lab on Tuesday night to get all of the correct dependencies for the open pose code installed on the board.  We were able to get our pose comparison code running on the TX2 by the end of the week.

This week I also began work on the user profiles and leaderboard.  I am currently setting up databases with SQLite to track the top 3 scores, as well as an individual  chart of scores for the user.

Next week, I plan to continue testing and debugging our pose comparison code on the TX2, as well as continuing with the user profiles.

Posted on

Zixuan’s Status Report for 4/3/2021

This week I continued working on the comparison algorithm and building the comparison metrics library. Since the library contains the key points’ coordinates of the standard poses, the workout duration, and the timestamps for the frames we will compare, I need to first select the frames, collect the data, and then put things together for each exercise. I also worked on the script that looks for the frame in the user video that has best match to the standard pose; in other words, the frame with the pose most similar to the standard one.

I think I am a little behind the schedule, but I will finish up the pose comparison part tomorrow and have my code ready for integration on Monday, thus I should be able to catch up. Our group will meet on Monday to put everything we have together on the board. I will try to modify/optimize the comparison algorithm after we have it tested on the board.

Posted on

Team Status Report for 4/3/2021

The most significant risk to our project is moving all of our software to the board. We will all meet to work on it together, and we are planning on dedicating as much time to this class as possible so we can finish installing OpenPose/our code on the board.

Currently, there are no changes to our schedule because we are in our catch-up week (still have until Monday). Once we get everything on the board, next week will be debugging and testing how our code works on the board.

Posted on

Sarah’s Status Report for 4/3/2021

This week, I started creating the UI. I created the basic workout and rest screens. The workout screen has the demo on the left (will be looped for the number of repetitions), and the live camera feed (from the our USB web camera) on the right. The rest screen only has the 15 second (or 1 minute) countdown timer.

 

I also created the static evaluation workout using 3 exercises from each category (1 from each difficulty level). The user will be prompted to try to complete 10 of each exercise.

I am currently on schedule (we’re in our catchup/buffer week).

Next week, I hope to add labels to the exercise/rest screens (current exercise, next exercise) and scale the demonstration gif to match the live video feed’s size. I also hope to design the home screen and add logic to handle events triggered by the user (pressing keys to pause/quit workout). I also plan on creating the stretch library (for cool down/after the workout), and add more exercises to each of the 3 existing exercise libraries.

Posted on

Maddie’s Status Report for 4/3/2021

This week I worked with the TX2 for OpenPose install and testing.  However, today I experienced a setback when I had to reinstall my virtual box and restart the process.  I am now experiencing delays with the Jetson SDK install (stuck at 41% despite how long I’m running it for).

I also filled out our legs and arms workout libraries to match the same amount of exercises that we have added to our core library.  We hope to continue adding to our workout library as we go, but we definitely have a solid number of exercises so far.

Next week the focus is going to be moving our code to the TX2, so I will be working hand in hand with my groupmates to get our project running on the board.