Posted on

Ray’s Status Report for 09/30/2023

We start on our implementation process this week and everyone gets to work on their respective sections of work. I started learning and writing UI for our system with TKinter.

I got through many of the UI elements over the week and I plan to planning to approach the image display, video display, and database functionalities tomorrow. Below is the video I will refer to.

I also tried to get openpose to work on my laptop, but I am still getting errors based on CMake versions. My plan is to make Openpose run on my laptop as soon as possible, hopefully in the next 2 days.

I also looked through whole_body_from_image.py and keypoints_from_image.py examples in the openpose official repository. Base on the whole body example, display a sequence of poses in order can be realized by reading each image , setting up the pose datapoints in Shiheng’s cost functions, and then waiting for the user to input a correct pose. The timing could be an issue to implement and I might need to look into the Python functionalities to realize them in our system. Meanwhile, static pose evaluation can be realized with the following code as a reference:

Still, through discussions on the design review we presented this week, we noticed aspects of our project that requires more detailed considerations. In particular, the way we want to convey our pose dectection results need to be intuitive enough for our users. Also, the time interval between each verbal instruction should be different from the interval between each pose detection. I will take these into account when designing the user interface next week and reflect them in the design report that is coming up next week.

My progress for this week is still on schedule, though work for next week might be heavier than regular weeks. I plan to work some time during the fall break if the schedule for next week turns out to be too packed, but I will put priority on planning out how the api and ui should communicate and decide on a design for the ui.

Posted on

Team Status Report for 10/07/2023

Week 10/7/2023: Continuing to revise design and initial project development.

On Wednesday, we finished our design review presentation, which we continued our discussion on our design during our Thursday night meeting. We were joined by professor Tamal who gave us valuable feedback on our current project and how we could improve our MVP framework to include more functionality. One of the things we adapted was the idea of evaluating users’ performance over a short period of time to understand the fact that they need to setup and transit into the position, eliminating the chances that they are doing a completely different posture but accidentally got a higher score due to the frames grabbed by our system, and provide an internal evaluation/scoring system for body postures. We still need to polish these ideas before next Friday, when the design report is due, before which we would need to quantify these thresholds and reflect the changes in our report.

On Friday morning, we received detailed feedback thanks to hardworking faculties on our design presentation. We believed that a detailed presentation was given but it seemed we still needed to clarify some issues that were not showcased clearly within our presentation.

The team worked throughout the week to perfect the details and narrowed down how we wanted to approach this project, which we determined that minor changes need to be made to our schedule. We have begun coding and incorporating systems gradually into our project, for which we already have Openpose algorithms working ahead of time thanks to the extra effort Hongzhe (Eric) put in to configure and run the system.

We’ll continue to work with faculty members in the following week on our design report and start data collection through imported poses we handpicked online from Taichi professionals, integrating Openpose API, and start developing the comparison algorithm. We currently have no plans to order any hardware equipment, and please stay tuned for our next update and design documentation.

 

ABET question:

  1.       Engineering Ethics: We highly respect the privacy of users due to the necessity of using cameras to capture body posture. It is possible that the camera could collect more personal info than posture, so we planned to make this a local application without the need for exchanging information with a cloud server. All collected data will be stored and evaluated locally without the need for connection to the internet, and users can choose to close the camera during the usage of the application if they feel necessary. (e.g. reviewing posture, attending to other businesses, or feeling not to practice)
  2.       Math model in comparing body postures: Using the idea of cosine similarity, we attempt to compare body postures captured from users to our standard preset provided by Taichi professionals. To account for differences in heights and shapes of people, we directly measure the joint angles of a person instead of directly approximating how similar the person is due to the differences of absolute positions of joints. Normalization principle could also be applied to postures to account for magnitude differences in vectors for varied body sizes.
  3.   Machine Learning model in OpenPose: In the posture recognition application OpenPose, we are able to use the trained convolutional neural network for recognizing the core body coordinates from the input video/image. Convolutional neural networks are based on simple batch processing, dot product of matrices and many techniques such as pooling and regularization to avoid outliers and overfitting. It utilizes all kinds of principles and formulas from Mathematics as logarithmic calculation to enable convolution and activation function. It also uses differentiation calculus for training the network parameters when back propagation.
Posted on

Shiheng’ Status Report for 10/07/2023

This week, I put my focus more on implementing the comparison algorithm and normalizing body sizes for better results which accounts for difference in body sizes. Abiding our Gantt Chart, here’s my progress:

Progress:

Since our pipeline from OpenPose is still under development, I constructed some datasets myself (purely random) to test my comparison algorithm using cosine similarity. Cosine similarity measures the angular similarity between vectors, making it ideal for assessing body orientations. Additionally, I will explore techniques to normalize body sizes to enhance the accuracy of these posture comparisons in the following week.

To facilitate comparison, each body posture is transformed into a vector within a multi-dimensional space. Each dimension within this space corresponds to a specific key point (in our case, joint) detected by OpenPose. For instance, if I am receiving from OpenPose output consists of 18 absolute positions, each posture is then represented as an 18-dimensional vector.

The implementation requires packages including numpy, normal Python environment and VSC for development. I used 3.11.5 in this case since 3.12 was just released a few days ago which could have compatibility issues with package supports. I’ll make sure to keep targeting the latest version for optimization and support of the packages.

 

Implementation on absolute position (Planned for next week):

To account for differences in body sizes and variations in the distances between body joints, it is imperative to normalize the posture vectors. The idea I have now is to normalize every person into my body size, which is around 5’10 and 170 lbs (need to be justified next week with other members of the group). This will be an add on to the cosine comparison idea to determine the absolute position of users. The idea of using absolute position eliminates the possibility that the user is doing a completely different posture and scores a high similarity due to the nature of cosine similarity. The normalization process involves dividing the coordinates of key points by reference length, which is typically the distance between two consistent points (two shoulders, the ratio between upper and lower body, the ratio between calf and thigh). This procedure scales all joints proportionally to this reference length, facilitating a relatively standard comparison.