Posted on

Hongzhe’s Status Report for 2023/12/09

This is the last official week of the capstone project course before we step into public demo and final report writing.

Over the past week, we first accomplished the final presentation to showcase the work, result and performance of our system. It is also a great experience to see how other teams has come along the way, and I am really interested in trying their products out in the following week. In the meantime, we were able to come up with more user testing, more detailed verification descriptions that we are planning to put into the final report. For example, about the question on how we are getting the percentages, we conduct tests on user data as well as our own testing data with numerous body images. For each image, there will be multiple key body angles that we keep track of, and that data adds up to calculating the confidence interval of our angle calculation system.

We will work on public demo, as well as a final report/video next week, and thank you, the staffs, for helping out through the semester.

Posted on

Team Status Report for 12/09/2023

The culmination of our project approached with the final presentation week, during which the team strategically allocated its efforts towards the meticulous creation of slides, thorough reviews of the presentation content, and collaborative endeavors. This period saw extensive discussions centering on crucial aspects such as testing plans, real-world applications, and in-depth data analysis, all with a primary focus on assessing the effectiveness of our developed system.

A significant facet of our recent efforts has been the intensive focus on comparative user testing, specifically aimed at measuring the accuracy of limb angles. This involved a dual approach, incorporating video training alongside our team’s system. As part of this process, ongoing data analysis is underway by Jerry and Ray, with the objective of extracting valuable information that will inform further refinement and optimization.

Our team’s productivity remained consistently high throughout the week, particularly in the preparation for the final presentation. Engaging discussions encompassed quantitative results derived from extensive testing, identification of potential areas of improvement, and careful consideration of user feedback. These discussions proved instrumental in shaping the final narrative of our project, ensuring a comprehensive and polished presentation.

Simultaneously, our focus extended to the backend processes of our project. Collaborative efforts, particularly with respect to finalizing backend functionalities, were undertaken in close partnership with Shiheng and Hongzhe. This phase involved meticulous optimization measures, enhancing overall system efficiency and responsiveness.

Noteworthy attention was also given to the refinement of design elements, with a dedicated effort to ensure optimal user experience. Collaborative discussions, particularly with Hongzhe, aimed at fine-tuning these elements, addressing any remaining concerns, and ensuring a seamless integration of design and functionality.

As the team progresses, we find ourselves currently on schedule, actively engaged in the critical phases of testing and evaluation. The imminent completion of the video and final report stands as a testament to our commitment to delivering key project deliverables with the highest standards of quality and precision. The collective efforts of the team position us well for the successful conclusion of this project.

Our team also wishes to express gratitude to all the faculty and TAs of the course for their continued assistance and guidance throughout the project.
Special thanks to our team’s TA Eshita and Professor Byron for meeting with us weekly, checking our progress, and providing constructive feedback throughout the semester.
ABET Question:

Unit Test on Frontend:
Functionality Testing to ensure the custom file upload feature functions as intended. Users are able to upload custom pictures. Widget Order Testing to verify that widgets associated with file upload are ordered correctly, user are able to rearrange and remove pictures.
UI Testing: UI elements and widgets work correctly under full screen resolution, and scales correctly.
Training Loop Testing: The training loop screen correctly showcased user image, skeleton, and reference picture.
Skeleton Drawing Testing: Correct skeletons are posed for both user and reference posture, previous issue mentioned in last week about scaling have been fixed.
File Storage: Local file storage is working correctly, both for prerecorded poses and reference poses.

Unit Test on Backend:
Angle testing: Justified angles captured by the Openpose system through the keypoints being passed into the backend. Used online protractor to measure angles compared to the calculation derived from keypoints.
Voice testing: Ensure voice module is robust and produces correct instructions as intended. Integrated throughout the testing process and accepted advice from users to improve instruction. Clear verbal instructions given with correct angles.
User posture testing: Different poses are being passed into the algorithm for testing, including incomplete postures, wrong postures. Correct angles and instructions are verified from feedback of group members and volunteers. Incorrect and correct limbs were clearly identified and passed to the frontend for drawing.
Correct person testing: When multiple people are involved in one frame, the algorithm is correctly identifying the user practicing the posture (most similar person).

Integration testing:
Testing on windows laptops, applications constructed by Kivy is able to launch and run correctly.
Pose sequencing: Functionality is working as intended, user is able to move on from one pose after doing it correctly. After finishing the sequencing, user will be presented with overall score of the sequence.
Parameter Adjustment Testing: Users are able to customize preparation time, move-on time, and tolerance angle for custom training. The sliders communicate with backend correctly to reflect the changes.
Time Trials: Timer are utilized to measure performance on different laptop setups from group members. User testing has come back with results of the app being slow, main bottleneck is the Openpose utilizing CUDA. Time requirements are within our set standards.
User Testing: Jerry collected data and did analysis on this part mainly. User experiences were measured, and bugs were found about the training loop, fixes were implemented to address the issue. Efficiency for the application increased after implementing the fix.

Posted on

Ray’s Status Report for 12/09/2023

The past week has been the final presentation week. I mainly focused on working with creating the slides with my team and filling out the final presentation reviews. I had a great time learning about all the useful and fun systems the other teams created.

Besides the final presentation, we had discussions among ourselves about the testing plan and real-world application of the application. We did some analysis on the data Jerry collected and checked how supportive the tests will be for showing the effectiveness of our system.

We also created the poster for next Monday’s public demo, which we believe will be a good opportunity to show case our application and collect some good user testing data. I managed some of the formatting and layout process for the poster.

Overall, we have completed all the functional implementations for the project. We are well on track finalizing our project, and for next week, I’ll work on the public demo and the final report, wrapping up the project as smoothly as possible.

 

Posted on

Jerry’s Status Report for 12/9/2023

This week I mostly focused on comparative user testing.  In our final demo, the professors expressed the desire to see user testing that measures the difference in accuracy of limb angles to the reference pose, with training with a video, and training with our system.

Users would test using a side by side of the video Ray gathered the reference poses from, and a webcam on the side to mimic a mirror, so they could look at their own pose and adjust.  I specified the users to train on the pose sequence of “repulse the monkey” in the video.  I had them train with the video for 5 minutes and then took a picture of the pose they felt they did the best at.  I would then have the users train on our system with a tolerance of 5 degrees for 5 minutes and use the latest training image and keypoints coordinates json file the system took for evaluation.

I fed the picture of the pose the users did with the video into openpose to get a json file of the coordinates, and then fed the json file into the comparison algorithm with the reference pose’s json file to get a list of similarities.  I did a similar thing with the json file of the users’ keypoints when training with our system.  I am still doing data analysis on the files and as such do not have an update yet to share.  We found that the users did better with the video than with our system when looking at an average of all of our similarities, however when we look at the similarities split up by joint angle type, we see that this is due to one main outlier: the angle between the right calf/shin and the right foot.  There are other joint angles that show large differences in favor of the similarity measurements of the user training without our app, but they are not nearly as dramatic as the angle between the right calf/shin and the right foot.

The full plot of the similarity differences can be found below.  The y axis is the similarities of the users joint angles training without our app minus the similarities of the users joint angles training with our app.  Thus a negative value, means the users showed better similarity to the reference pose when training without our app and a positive value means the users showed better similarity to the reference pose when training with our app.

Link to graph: https://docs.google.com/presentation/d/1GJe1FTYBJIslWOp5YyHfYAI1yyZRDDB5vdt_yO-MKDY/edit?usp=sharing 

I also helped make the poster and final presentation this week as well, including making a short gif of the custom upload pipeline in action.

We are well on schedule as we are doing testing right now and evaluating our system.  Nothing remains to be done with our project in terms of functionality.

In terms of deliverables we plan to finish making the video and writing the final report to turn in on time.

ABET Question 6:

In terms of tests we did, I did numerous unit tests on my custom file upload in terms of testing the functionality of the system and making sure widgets were getting ordered correctly, and that the custom upload feature was robust under continuous use.  In terms of full system testing, our team did many full system tests during the integration phase of our project, and user testing and comparative user testing also doubled as a full system test.  In terms of our tests, we found that users overall felt the app was a bit too slow, I noticed one user who mentioned it was very slow, ran into a bug in our system where openpose was continuously running in the training loop, even when the user exited the training screen.  I fixed this bug and our user scores for the speed of our app went up.

 

 

Posted on

Shiheng’s Status Report for 12/09/2023

This past week has been incredibly productive as I dedicated a significant portion of my time to preparing for an upcoming presentation. The focal point of our discussion centered around the quantitative results derived from the extensive test trials we conducted. Through a thorough analysis of the gathered data, we were able to draw valuable insights that formed the backbone of our presentation.

One key aspect of our discussions revolved around potential areas of improvement, which we identified through meticulous examination of user feedback obtained during the testing phase. This feedback proved invaluable in shaping our understanding of user experiences and guiding us towards refining the functionalities of the project.

Simultaneously, my attention was devoted to finalizing the backend processes of our project. Collaborating closely with Hongzhe, we delved into intricate details to ensure the seamless integration and optimal performance of the backend. Several optimizations were implemented, enhancing the overall efficiency and responsiveness of the system.

In addition to the technical aspects, I engaged in detailed discussions with Hongzhe regarding the design elements of our project. Fine-tuning the design for optimal user experience was a priority, and we worked collaboratively to address any remaining concerns on the backend.

Posted on

Hongzhe’s Status Report for 12/02/2023

This is the last week before the final presentation and we have mostly done the final tuning of the application development process together as a team.

I participated in the demo for final check to the course faculty, helped design the user testing questionnaire. I am also in charge of the backend performance testing (time consumed for each portion of the code that processes user data and counted into the waiting time), and worked with Roland in designing backend accuracy/performance testing plan. I am also the model for the accuracy plan which would be shown later in the presentation.

In the future week, I am planning to help conduct more user testing and gather opinions of the functionalities and parameter tuning of the system. We will also focus on preparing the final paper/poster work for the course together as a team.

Posted on

Jerry’s Status Report for 12/02/2023

This week I took the lead on user testing for our app.  I created a google form to test for user’s satisfaction with various aspects of our app such as speed and usefulness of voice generated instructions.  In total I had 5 users try out our application.  I had users try the first pose, and had users select the “repulse the monkey” pose.  I had the first user test on an angle tolerance of 4, 10, and 15.  For the other 4 users I had them test on an angle tolerance of 15 and 20 degrees.  After the first user tested, I found a bug where the training loop would continue running even after leaving the training page.  I fixed this bug before testing with the other users, so the first user’s review is biased due to it being on a slower version of the system.  Some commonalities from the feedback we were given is that users overwhelmingly preferred an angle tolerance of 20 degrees, they all found the mirroring of the camera unintuitive, but our app on the whole was fairly usable.   Below is a picture of the responses we received to 2 of the questions in our google form:

I also helped to complete the final presentation slides with my teammates so we can be ready for our final presentation next week.

 

I believe I am on schedule as we have completed some of the verification and user testing needed for our project that we wanted to complete this week.

In terms of deliverables next week, I plan on continuing user testing as well as working on the design report and poster.  I also anticipate working on some of the issues mentioned in our user testing if my partners need help completing some of the feature improvements users would like to see.

Posted on

Ray’s Status Report for 12/02/2023

My team and I made considerable progress on the project in the past two weeks. Except for some minor bugs, our application is functioning properly in general, and we are also actively optimizing our application to make it more convenient for our users.

To briefly summarize my work over the two weeks:

a. Training page posture sequencing works

Last Saturday and Sunday, I worked on the training page posture sequencing based on the implementation plan I created in advance. Since I finished the posture drawing functionality ahead of the plan, I implemented the posture sequencing functionality as well.

Now, for a Taichi posture sequence, the training page go through all subposetures in the sequence so that the users can actually train themselves on the full Taichi posture. When the user correctly performed a subpose, the training will automatically switch to the next subpose.

I also created a new configurable parameter “move-on time” for the users  in the setting screen to control the interval between subposes. It is different from the “preparation time” parameter, which is time interval between the moment start button is pressed and the evaluation of the first subpose.

Below is a image of the new trianing page (Image provided by Shiheng (Roland)).

b. Result page implemented

When the training of a Taichi posture ends, training screen now switches to a result screen, showing average score, total time, and a random tip on how to use our application correctly.

c. Tutorial page implemented

I created a new page showing a tutorial on how to use our application correctly. It is accessible from the menu page. Below is the tutorial page.

d. Minor fixes for better user experience

Selection screen has much larger pose item in order to make the image preview easier to view. Back buttons in all screens are enlarged. I also made a few small fixes on training screen timer display issues.

In general, my team and I am back on the schedule and we are promptly finalizing our project. Next week, I will fix the bugs and issues I found during testing and also brainstorm on what extra feature I want to add to our application.

Posted on

Shiheng’s Status Report for 12/02/2023

For this week, I mostly focused on fixing minor bugs along with integration issues faced with testing out the application with rest of the group. Guidance was improved for users not fitting inside the frame of the camera and about correcting positions. I worked with Ray about issues I found in comparing skeletons about missing joints and scaling of the reference picture. Eric also assisted me in rerunning the openpose script on cropped picture for better performance in the application.

Issues were also found inside our voice module causing unintended termination of the application, which were found caused by pygame module. For the weekend and upcoming week, I will be preparing for the upcoming presentation and doing testing on my backend module along with Eric for justifications and evaluation purposes.

Posted on

Team Status Report for 12/02/2023

For our Taichine project this week, we finished all planned functionalities in our MVP and were mostly focused on polishing the final details of the projection prior to the testing and verification phase.

During Monday and Wednesday meetings, the team focused on testing on the front end along with demos of the project to faculty members. In the process, bugs were found throughout the test when group members were trying out our final product.

Shiheng and Ray worked on the backend and integration scripts about cases where users were not correctly positioned in frame and when nobody is detected in the camera. Previous solutions were not comprehensive to users and more for the purpose of debugging, which now have been replaced by voice instructions and redrawing of the skeleton to guide users to improve their current posture. Shiheng also worked on implementing voice instructions and fixed failures where the pygame module will cause unintended termination of the training. Ray improved UI elements on the frontend and changed scaling algorithms for better showcase of skeletons.

Hongzhe (Eric) worked on cropping and regeneration of the openpose data on the Taichi pose pictures to accommodate the frontend changes of picture frame, during which he also helped figuring out the bugs encountered on the frontend on skeleton drawing for the reference pose and user input. Eric also suggested that logic improvements on the backend to Shiheng about pose priority, where fixes were implemented upon.

Jerry worked more on the custom pose implementation side and focused on file storage structures for sequencing and stepping through the images for prerecorded poses. Now both pipelines follow similar naming trends and users are more easily directed to figure out where the custom poses are stored. Also, some button logics were revised on the frontend by Jerry to improve performance of the application and remove redundancy.

We will focus on testing, verification, and validation of our project for the weekend and for the week prior to final demo. In addition, we will work simultaneously on other scheduled matters including slides, posters, and videos to accommodate for more flexibility during the final week.