Posted on

Shiheng’ Status Report for 10/28/2023

My work this week mainly focused on comparison algorithms. Using the json data generated from Openpose thanks to Eric, I was able to craft my python script for comparing two different postures. There was some confusion at the beginning since there were some extra outputs than I previously expected, but it was figured out after communicating with Eric and looking at the documents from Openpose. The keypoints were narrowed down and point representing positions like eyes and ears were eliminated to improve the accuracy of determining overall body posture.

Only point 0-14 are used in judging posture right now for efficiency, I’ll see how certain keypoints (on the feet) impact the correctness of posture in testings following the implementation.

Using packages like numpy and json, I was able to pull files from the locally designated folder and compare the sets of data within. I added a few steps to process the data including the mentioned reduced datapoints, but also reforming vectors from the given limb datapoints. The reformed ‘limbs’ were then passed into the cosine similarity comparison. The response of the algorithm was quick, and results were easy to understand. The way we wanted to display the results and the mechanism is still to be determined, so I will display a raw output below from my algorithm to showcase one example of comparison based on two trials of a same image but which I did some minor changes in the output position of the upper limb of the second image.

The next steps will be to connect the comparison algorithm with the TTS engine to invoke vocal instructions and move on the integration steps accordingly.

Posted on

Ray’s Status Report for 10/28/2023

For the past week, I have been learning and writing KV language as well as adjusting the current prototype framework that we have. I changed a lot of widgets’ configurations so that they are now aligned according to the size of the application window, and I also successfully got the setting page to work.

My next important task is to create the training page, which our group comes to an agreement on the design. We now want to display the user’s footage on the top left corner, the reference posture on the top right corner, and the two skeletons at the bottom of the application window. The skeletons will be drawn with the same body part lengths, but the angles between different body parts will depend on the reference pose and the user’s attempt at imitating it. We believe this will help users know how to correct their mistakes the most intuitively.

Also, with help from Eric, Openpose is finally running on my laptop! While the framerate is very low, it should be enough to let me start testing out the openpose api in the official documents. I plan to try them out next week and see how things work for our overall integration.

Also, my teammates and I also looked into the ethics aspect of our project. We met and discussed this week to see if there are any extra design choices necessary to make our project ethically robust. It turns out the parameter adjustment option that we are offering to our users on the setting page has already resolved some of the most important ethical issues concerning our project.

I believe I am currently a little behind schedule since the integration process is expected to start next week, but I believe I can catch up early next week as I have clearly identified and prepared for the tasks left for the frontend part of our project.

Posted on

Team Status Report for 10/28/2023

During the past week, our team has been diligently working on various aspects of our project and collaborating to work the magic.

 

Ray and Jerry are working on the UI design using Kivy about functions like image uploading and displays. They will continue to cooperate with other members of the team to integrate functions like Openpose and voice engine into the application. This will play a pivotal role in ensuring a user-friendly and visually appealing experience for our application compared to the more naïve implementation of Tkinter.

 

Eric has successfully integrated Openpose into our system, enabling it to accept image inputs and generate JSON outputs for the comparison algorithm for Shiheng. This is a pivotal step in our project, as it provides the foundation for our comparison algorithm and more detailed instructions on body postures. We’ll look at how we want to display Openpose feedback integrated with camera images to the user in the following weeks.

 

Shiheng has implemented a cosine similarity algorithm for comparing key points that represent body posture. This algorithm will allow us to quantify the similarity between different body postures, providing a strong basis for evaluation and giving out instructions. We can analyze and compare the alignment and positioning of individual limbs, offering a more detailed assessment of body posture.

 

We also discussed issues on ethics after we specified more about project details on our Friday meeting, during which we first exchanged opinions about the reading and answers we made on the project, then proceeded to argue more on privacy, autonomy, and potential issues of misusing the project.

 

Our team is making solid progress, and we’re on track to deliver our comprehensive Taichi instructor system. We’ll continue working on these components, work on feedbacks from our design review report, and aim to achieve our project milestones in the coming weeks.