Category: Team Status Reports

This week we continued to work on integration and started working on the demo video.

The most significant risk is that the accuracy for some core exercises is lower than we want.  We decided to remove some exercises that are hard to detect (such as elbow to knee) from the workout library.

We will work on the final video and the report next week.

This week we mainly worked on the integration of our project. Now we can run the pose comparison with the UI.

The most significant risk for us right now is the successful integration of the entire system on the board. We need to resolve some issues caused by our different configurations. We hope to have everything put together next week and test the entire system.

Next week we will work on the final presentation and continue to integrate our system.

A risk to our project is not being able to run the UI on the board. It could possibly run differently on it. Another risk is the pose comparison being too slow or the UI lagging once we get both on the board. To mitigate any problems from integrating all of our software on the board, we will get it on the board by Monday so we can start testing how everything works together.

One change was made to our materials. We bought a second camera (same model as the one we already have) because we ran into issues with recreating bugs (we encountered on the Xavier) locally on Zixuan’s laptop. We found that the difference in frame rates of the laptop camera and web cam on the Xavier was affecting the behavior of the pose comparison code. This only increased our project budget by $30.50.

The most significant risk to the success of our project is the TX2 board freezing or failing to execute our pose comparison code. It used to run once in 6 tries, but now it runs half the time. The failures are caused by memory errors. We think it might be using too much memory because we keep getting memory notifications. If we restart the board, we found that the memory errors are less likely to happen. But we want to find a more permanent fix.

We do have a backup plan of running everything on a laptop, but we don’t think we’ll need to resort to this backup plan.

 

The most significant risk to our project is moving all of our software to the board. We will all meet to work on it together, and we are planning on dedicating as much time to this class as possible so we can finish installing OpenPose/our code on the board.

Currently, there are no changes to our schedule because we are in our catch-up week (still have until Monday). Once we get everything on the board, next week will be debugging and testing how our code works on the board.

One suggestion we received was to add a live video of the user so they can see themselves during their workout. However, we were thinking this feature wouldn’t be necessary because of the group we are creating this product for. People are mostly working out at home right now, and using YouTube workout videos, but they mostly aren’t watching themselves do the exercises. I don’t think many home workouts are done in front of a mirror (at least according to people I’ve asked). We’re also concerned about how much latency this feature would add to our product.

Even if they did do the workouts in front of a mirror, we think that the ability to watch themselves during the workout will distract the users. To look up at the live video, the user will need to use incorrect form. This is something we don’t want to encourage the users to do, because incorrect form can lead to injuries over time because of strain caused to certain areas of the body. Also, during workouts, the users should be focusing on engaging the muscle groups that a particular exercise is targeting.

Maybe as a compromise, we could add an overview of the generated workout so that users can learn how to perform the exercises before the actual workout starts. This part of the product could be controlled by the user; as in, they could choose when to see the next exercise (press the next arrow key) so they can take as much time as they need to learn one exercise before seeing the next. This would also allow users to skip exercises they already know, so they can focus on learning only new exercise forms. In this, we could provide the same looped videos so the users can copy them. The users would be able watch the demo video, find a mirror (or we could provide the live stream video here), and try to mimic the demonstrator’s form correctly.

This week we received feedback from our design review presentation. One of the most significant risk was comparing the poses at different speeds. Our plan was to select specific frames from the user input based on time and compare them with the poses from the workout library, which would not work if someone is doing the exercises much faster/slower. We are looking in to dynamic time warping as suggested by Prof. Savvides, and trying to apply that in our project to correct what we compare by looking for the best alignments between time series.

We also need to figure out more quantitive and detailed ways to measure accuracy of pose alignment and comparison. We planned to test it with different body types and poses, and check whether it can align the points and distinguish the poses correctly.

We started working on our design review report.

We decided to use a Jetson TX2 Xavier board instead of a Jetson NX Xavier or Jetson AGX Xavier. This is because The 18-500 parts inventory had both the AGX and TX2 boards from previous years. We didn’t want to buy another board if there were already 3 usable boards available. We originally wanted to borrow an AGX, but other groups requested the 2 in inventory earlier than we did, so we weren’t able to borrow them. The inventory still had a TX2, so we submitted a request to borrow that instead, and were able to do so.

Borrowing the boards instead of buying a new one allowed us to stay exactly on schedule. If we bought an NX, then we would’ve been at least 2 days behind because the shipping time is 2 days. It also decreased our costs, so we’re only paying for a camera currently.

To adjust to the board we chose, the TX2, we decided to use a different library: Tensorflow OpenPose. This library was able to analyze some sample images (on Zixuan’s laptop) much faster than the original OpenPose was. While she was testing OpenPose, she was able to verify that the images we took from YouTube videos are usable with Tensorflow OpenPose.

This week, we also worked on our design presentation slides and tried to incorporate the feedback we got from the proposal presentation. So, we added more pictures/visuals, and a testing plan (not actually in the slides, but will be talked about). We’re currently still working on our reasoning for our metrics, but this should be done by the time we present.

Currently, the most significant risk to the success of our project is that we are already behind schedule (based on our Gantt chart). We wanted to have our board by this coming Monday, but we haven’t ordered a board yet because we are potentially changing our first choice board from the Xavier NX to the Xavier AGX. This is because the AGX is available according to the inventory list, so it’s now within our budget. The only thing that is preventing us from finalizing our decision is that we haven’t been able to find as many resources for using OpenPose on the AGX as we have been able to find for using OpenPose on the NX.

We’re planning on finalizing which board we want to use by Sunday so we will still have time to get an NX approved by our TA and ordered by Tuesday. If we end up choosing to use an AGX, we won’t be very behind schedule because we could pick that board up on Monday.

The biggest risks to the success of our project are our hardware performance and overall latency (from hardware and software). We made backup plans to address both concerns, in case our initial plans don’t meet our performance goals.

Our current plan is to use the Xavier NX, though our backup plan for hardware is to use a Jetson Nano, which is cheaper than a Xavier NX or AGX Xavier. If that doesn’t work well, or our budget doesn’t allow, our next backup will be our laptops.

Our backup plan for latency (because we want to perform real-time processing of images of the user) is to end workouts with a “Score loading” screen, to provide some extra buffer time for our scoring/image processing algorithms to finish.

We divided our project into tasks (and subtasks) and divided them up among ourselves. We reflected this division of labor in our Gantt Chart.