Tag: Status Report

During the past week, our team has been diligently working on various aspects of our project and collaborating to work the magic.

Ray and Jerry are working on the UI design using Kivy about functions like image uploading and displays. They will continue to cooperate with other members of the team to integrate functions like Openpose and voice engine into the application. This will play a pivotal role in ensuring a user-friendly and visually appealing experience for our application compared to the more naïve implementation of Tkinter.

Eric has successfully integrated Openpose into our system, enabling it to accept image inputs and generate JSON outputs for the comparison algorithm for Shiheng. This is a pivotal step in our project, as it provides the foundation for our comparison algorithm and more detailed instructions on body postures. We’ll look at how we want to display Openpose feedback integrated with camera images to the user in the following weeks.

Shiheng has implemented a cosine similarity algorithm for comparing key points that represent body posture. This algorithm will allow us to quantify the similarity between different body postures, providing a strong basis for evaluation and giving out instructions. We can analyze and compare the alignment and positioning of individual limbs, offering a more detailed assessment of body posture.

We also discussed issues on ethics after we specified more about project details on our Friday meeting, during which we first exchanged opinions about the reading and answers we made on the project, then proceeded to argue more on privacy, autonomy, and potential issues of misusing the project.

Our team is making solid progress, and we’re on track to deliver our comprehensive Taichi instructor system. We’ll continue working on these components, work on feedbacks from our design review report, and aim to achieve our project milestones in the coming weeks.

For the week prior to the fall break, our group mainly focused on finishing and polishing the Design Review document. We split up the work evenly, with every team member taking up some part of the writing and filling in the part of the subsystem they are responsible for in the following developments. We were able to find existing journals and research to justify our design requirements, provide details for our system specs, and think of alternative approaches if some parts of the system would go wrong. The writing of the design documentation aided us to explain a lot more details not mentioned in our design review presentation due to time constraints (since we need to fit contents of two presentation into one as we redesigned our project after the proposal presentation), providing faculty a better picture of the project and helped clarifying details among group members.

While enjoying our fall break, we also made some progress in implementing some of the subsystems.

Thanks to Hongzhe (Eric), the Openpose posture detection is now working with imported images (handpicked high resolution from online video source) which we are going to use as reference postures and baseline in evaluating user poses. As we described inside our design documents, these images will be transformed into json files with key points of postures in 2D that will be passed into the comparison algorithm for calculation of the differences.

Shiheng worked on the Text-to-Speech engine, where it could take in text instructions generated from a comparison algorithm and pass them into the voice engine to generate real time instructions. The time required for the voice engine to generate output files is low (< 0.5 seconds for an average length instruction) and the output voices are clear and easy to understand. We will continue to look into voice engine outputs and determine the best way to present these vocal instructions to users.

On the frontend, Jerry looked into the project considering several Python packages for better UI and user considerations. He determined that the Kivy package will provide us with lots of widgets that could be easily implemented and a much better interface than Tkinter which was what we planned to use originally.

Ray focused on learning the kivy language and creating the prototype frontend application. He created a prototype page for the main menu and the pose selection page. He is also working on the screen manager feature of Kivy to support page switch in the application and expeirmenting on openpose data interfacing with Kivy UI.

Week 10/7/2023: Continuing to revise design and initial project development.

On Wednesday, we finished our design review presentation, which we continued our discussion on our design during our Thursday night meeting. We were joined by professor Tamal who gave us valuable feedback on our current project and how we could improve our MVP framework to include more functionality. One of the things we adapted was the idea of evaluating users’ performance over a short period of time to understand the fact that they need to setup and transit into the position, eliminating the chances that they are doing a completely different posture but accidentally got a higher score due to the frames grabbed by our system, and provide an internal evaluation/scoring system for body postures. We still need to polish these ideas before next Friday, when the design report is due, before which we would need to quantify these thresholds and reflect the changes in our report.

On Friday morning, we received detailed feedback thanks to hardworking faculties on our design presentation. We believed that a detailed presentation was given but it seemed we still needed to clarify some issues that were not showcased clearly within our presentation.

The team worked throughout the week to perfect the details and narrowed down how we wanted to approach this project, which we determined that minor changes need to be made to our schedule. We have begun coding and incorporating systems gradually into our project, for which we already have Openpose algorithms working ahead of time thanks to the extra effort Hongzhe (Eric) put in to configure and run the system.

We’ll continue to work with faculty members in the following week on our design report and start data collection through imported poses we handpicked online from Taichi professionals, integrating Openpose API, and start developing the comparison algorithm. We currently have no plans to order any hardware equipment, and please stay tuned for our next update and design documentation.

ABET question:

1.       Engineering Ethics: We highly respect the privacy of users due to the necessity of using cameras to capture body posture. It is possible that the camera could collect more personal info than posture, so we planned to make this a local application without the need for exchanging information with a cloud server. All collected data will be stored and evaluated locally without the need for connection to the internet, and users can choose to close the camera during the usage of the application if they feel necessary. (e.g. reviewing posture, attending to other businesses, or feeling not to practice)
2.       Math model in comparing body postures: Using the idea of cosine similarity, we attempt to compare body postures captured from users to our standard preset provided by Taichi professionals. To account for differences in heights and shapes of people, we directly measure the joint angles of a person instead of directly approximating how similar the person is due to the differences of absolute positions of joints. Normalization principle could also be applied to postures to account for magnitude differences in vectors for varied body sizes.
3.   Machine Learning model in OpenPose: In the posture recognition application OpenPose, we are able to use the trained convolutional neural network for recognizing the core body coordinates from the input video/image. Convolutional neural networks are based on simple batch processing, dot product of matrices and many techniques such as pooling and regularization to avoid outliers and overfitting. It utilizes all kinds of principles and formulas from Mathematics as logarithmic calculation to enable convolution and activation function. It also uses differentiation calculus for training the network parameters when back propagation.