Carnegie Mellon ECE Capstone, Spring2021 || Maddie M., Sarah T., Zixuan Z.
This week we continued to work on integration and started working on the demo video.
The most significant risk is that the accuracy for some core exercises is lower than we want. We decided to remove some exercises that are hard to detect (such as elbow to knee) from the workout library.
We will work on the final video and the report next week.
This week, I downloaded the latest version of the UI onto the board. I worked with Maddie to fix an “illegal instruction (core dumped)” error the board kept producing. It ended up being a NumPy error. We had to downgrade the version. But now the UI works on the board. It’s integrated with the pose comparison, scoring, and accounts now. Zixuan was able to get the UI to show while the pose comparison runs.
I was also able to finalize the exercise library.
I recorded a bunch of demo clips this week, too. Maddie will crop and stitch them together for the demo video. We also finished our poster.
We’re on schedule. We just need to finish the video by Monday night. Then, next week, we’ll just need to do the public demo on Thursday.
This week, I downloaded all the dependencies necessary to run the UI on the Xavier board. The code didn’t run correctly on the board, but it wasn’t a dependency issue. I also had to resize the camera feed frames because the default size from the webcam was different from the default size on my laptop.
I’m behind schedule because I didn’t think I was going to be sick for that long after getting my second COVID vaccination shot.
Next week, I hope to download the new UI code on the board. Zixuan modified it so the UI could run at the same time as the pose comparison code. I also hope to finalize the exercise library.
This week we mainly worked on the integration of our project. Now we can run the pose comparison with the UI.
The most significant risk for us right now is the successful integration of the entire system on the board. We need to resolve some issues caused by our different configurations. We hope to have everything put together next week and test the entire system.
Next week we will work on the final presentation and continue to integrate our system.
A risk to our project is not being able to run the UI on the board. It could possibly run differently on it. Another risk is the pose comparison being too slow or the UI lagging once we get both on the board. To mitigate any problems from integrating all of our software on the board, we will get it on the board by Monday so we can start testing how everything works together.
One change was made to our materials. We bought a second camera (same model as the one we already have) because we ran into issues with recreating bugs (we encountered on the Xavier) locally on Zixuan’s laptop. We found that the difference in frame rates of the laptop camera and web cam on the Xavier was affecting the behavior of the pose comparison code. This only increased our project budget by $30.50.
This week, I added the evaluation workout to the UI loop. Now, it comes directly after the instructions screen. Once the user finishes the evaluation workout and is show their score, it’ll take them to a workout instructions screen (to inform them that the next workout will be a personalized based on the score they received on the evaluation workout). Then, they can choose when to start that workout.
I also made the rest screen look the same as the exercise gif screens. So, now all of the screens have the demo gif on the left and the user live video feed on the right. The label of the current exercise is on the top left corner and the label of the next exercise is on the top right corner. On the rest screen, there is also a label in between the current and next exercise labels that displays the rest countdown.
Lastly, I created a help screen that lists the quit, return-to-home, and pause commands. This help screen can be accessed from anytime in the UI except for the actual workout. So, the user can press “H” on their keyboard for help if they are on the home screen, any instructions screens, exercise preview screens, and scoring screens. I only allow the user to press quit the entire UI program, return to the home screen, and pause their workout once the workout is running. If the user isn’t doing their workout and goes to the help screen, they can press enter to return to where they left off.
I’m on schedule (we’re currently in our buffer time).
I’m planning on uploading the UI code to the Xavier board so I can integrate it with the pose comparison for scoring. I also am planning to add more exercises to our exercise library so we’ll have at least 100 exercises.
The most significant risk to the success of our project is the TX2 board freezing or failing to execute our pose comparison code. It used to run once in 6 tries, but now it runs half the time. The failures are caused by memory errors. We think it might be using too much memory because we keep getting memory notifications. If we restart the board, we found that the memory errors are less likely to happen. But we want to find a more permanent fix.
We do have a backup plan of running everything on a laptop, but we don’t think we’ll need to resort to this backup plan.
This week, I added labels to the exercise and rest screens. The labels are of the current exercise name (or “rest” if rest) and the next exercise name. I also made the demo video slightly larger to better match the camera feed’s size on the screens. I found a stretch video for the end of each workout, I just need to add it to the UI. I will be playing only a portion of it. I also started creating a workout preview screen before the actual workout starts (so users can familiarize themselves with the exercises in the workout).
This week, I also met with Maddie (on Sunday, Monday, Wednesday) to help set up the TX2 and download the necessary libraries/code onto it. For some reason, the installation of the NVIDIA package froze around 40% for her, and when we tried it on a lab computer, the virtual box couldn’t connect to the CMU wifi so we weren’t able to start the installation of the NVIDIA packages for the board. We eventually got the installation to work when I installed the virtual box on my laptop. I tried to take the board home and finish the installation there, but the board couldn’t find my wifi, so I finished it the next day with Maddie.
I am on schedule.
I hope to finish the workout preview before the interim demo.
The most significant risk to our project is moving all of our software to the board. We will all meet to work on it together, and we are planning on dedicating as much time to this class as possible so we can finish installing OpenPose/our code on the board.
Currently, there are no changes to our schedule because we are in our catch-up week (still have until Monday). Once we get everything on the board, next week will be debugging and testing how our code works on the board.
This week, I started creating the UI. I created the basic workout and rest screens. The workout screen has the demo on the left (will be looped for the number of repetitions), and the live camera feed (from the our USB web camera) on the right. The rest screen only has the 15 second (or 1 minute) countdown timer.
I also created the static evaluation workout using 3 exercises from each category (1 from each difficulty level). The user will be prompted to try to complete 10 of each exercise.
I am currently on schedule (we’re in our catchup/buffer week).
Next week, I hope to add labels to the exercise/rest screens (current exercise, next exercise) and scale the demonstration gif to match the live video feed’s size. I also hope to design the home screen and add logic to handle events triggered by the user (pressing keys to pause/quit workout). I also plan on creating the stretch library (for cool down/after the workout), and add more exercises to each of the 3 existing exercise libraries.