Tag: Status Reports

Current Progress:

• Tested the acrylic two way mirror with the monitors. Through the testing, we realized that the acrylic is a much better solution that the mirror film and so decided to stick to using this for the hardware.
• We spent time doing research into the wooden frame construction and the different tools we have at CMU (like techspark and what is needed to do wood working there) to build the frame. We are now designing the wooden frame to build next week.
• We have had extensive ideation sessions on how to map the camera coordinates to the mirror coordinates, which included a lot of trigonometry and physics for mirrors. We now have a much better idea of how we want to implement this and have broken them down into sub tasks.
• We have written a function to calculate the amount by which an object appears smaller as they move away from the mirror, which we will use for understanding which parts of a body is visible on the mirror.

Next Steps:

• Since we only have one work session scheduled during class time due to the ethics discussion, we are planning to meet 2 more times throughout the week to discuss our progress in our respective sections
• We divided out the work into actionable sections that should be completed by the end of next week
  • Youssef: Working on how to render the error corrections (highlight/circle nodes with openCV or using tkinter)
  • Sruti: Working on the Camera to Mirror mapping by scaling different nodes and transforming the coordinates using a camera projection matrix
  • Ankita: Working on planning out the hardware assembly, ordering parts, and potentially creating a CAD model of our product

Challenges/Timeline

• One major challenge is to complete the implementation of our mirror mapping methodology per our outlined approach and verify that it works as expected, tuning as needed
• We also need to complete the pose correction rendering and ensure that users can suitably understand their errors and correct based on these suggestions
• We are on track with our original timeline, but are beginning to have tasks with sequential dependencies, particularly on the hardware construction of the mirror itself. We need to have this completed in order to most accurately test the mirror mapping work and implementation as well as the final pose correction rendering

Current Progress

• Pose detection and correction sub-section completed: Completed pose comparison algorithm for all the poses
• Tested and tuned parameters of five out of the six yoga poses
• Received the monitors and two way mirror sheet. Tested the smart mirror with the mirror sheet and monitors. The results are pretty good, we can see the monitor’s display quite clearly through the two way mirror. However, since the sheet is thin, the reflection of the person gets blurred as the person moves further from the mirror. So, while we think this is a good back up option, we want to try the acrylic mirror and pick the best of the two. We ordered the acrylic mirror which we will be able to test after spring break and finalize our hardware.

• The pose correction algorithm can identify which parts of the body have errors and what needs to be done to fix them, which will help the error rendering sub-system.

Challenges

• The major upcoming challenges our team will face are related to translating the corrections generated from our completed Pose Correction component into visually and auditorily helpful corrective suggestions for users of the final product.
  • We currently associate each possible pose error with a text description on the backend but will need to find an extendable and flexible way to translate these into the visual and auditory feedback users receive to make sure the use case/design requirements addressing usability are satisfied
• We also need to complete the actual hardware integration and try out using the higher-quality acrylic mirror

Timeline

• Our original timeline expected hardware assembly for the mirror itself to be completed by now, but using one of our scheduled-in slack weeks before spring break, we are still on track with our original timeline.
• We should complete hardware setup of the mirror and camera by the end of the week and begin working on the mirror-camera coordination as the next major task for the following week

New tools for Sub-Systems

• Mirror Hardware System: Table saw, Wood glue, Nails, Sandpaper, Drill, Framing square
• Mirror UI: Magic Mirror library, tkinter

Next Steps

• The acrylic mirror should arrive within the next week, so we are aiming to finish the hardware assembly for the mirror.
• The rest of the poses for our Pose Correction component still need to be tested with the parameters for the Downward Dog pose especially needing a lot of tuning as the angles we specified are slightly too strict.
• The Mirror UI and mapping component is our next major software component after we wrap up the Pose Correction algorithm and according to our Gantt chart, we are planning to start working on the error display component next week.

Progress Updates:

• We continued working on the Pose Correction algorithm and completed the functions to check whether the current pose is within the defined threshold of the reference pose (as determined by extracting the notable node configurations from the yoga instructor’s ideal images)
• We researched different monitors that we could potentially use to construct a full-length mirror and decided to use a seamless, dual monitor set-up resulting in a final length of 48 in, which we determined was sufficient enough to account for most heights
  • We decided to use the flexible 2-way mirror sheet instead of the acrylic as an initial prototype as it was cheaper, thin (so the LCD screen would shine through), and had a variable length (as we were initially not sure which monitors we would end up using).

Risks:

• The primary risk so far is the Mirror hardware construction as we are not sure whether the 2 LCD monitors will be able to stick together using just the wooden frame or if additional measures will need to be taken
• If the wooden frame is not sufficient enough, we could also attach both monitors to an external piece of wood and overlay the mirror on top of that subassembly.

Changes to schedule and team work:

• No other schedule changes have been made and we will attempt to finish our backlogged tasks this week
• The pose correction is our top priority and all teammates are working on that right now but we will continue with our task split up after this
• There are no issues with our teamwork and accountability

Next Steps:

• Continue testing and refining the threshold parameters for whether the current pose is within error margins and can be classified as correct
• After receiving feedback on our Design Presentation, we will adjust our verification and validation procedures to create more quantitative and robust checks to ensure our design meets our stated requirements
• Contingent upon our materials arriving this week, we will start assembling the wooden frame encapsulating our LCD screens, attach the LCD screens together, and apply the 2-way mirror sheet to complete our hardware assembly
• For software, we will completely finish the Pose Correction algorithm and start testing it at various angles and distances to ensure it accounts for those discrepancies

Progress Updates:

• Had a meeting with the yoga instructor we are working with to get ideal poses and learn about typical errors and metrics to correct yoga poses
  • Collected reference pose images for all the 6 poses we want our system to have
  • Currently working on analyzing the poses to determine which nodes and what angles should be used for pose comparison. We are also computing pose detection values and storing them so it does not need to be recomputed each time.
  • Yoga expert helped us understand additional error signals used in in-person instruction that should be present in our pose correction system, most notable of these being the concept of alignment along axes on a person’s pose (for example, arms and shoulder should be collinear)
• Implemented library for Pose Correction component to compute:
  • Euclidean angles given 3 joints on reference pose
  • Distance between 2 joints (to check if they are in contact)
  • Check if all the required joints are visible (to make sure they are in frame, and that there joints are not one behind the other)
• Developed our design review presentation:
  • Fleshed out our system’s design
  • Better understood how to implement sub-systems
  • Practices and created slides

 Our Major Risks so far include:

• Monitor and hardware acquisition: We are hoping to acquire 2 monitors to construct a full-length mirror, but are not sure how to acquire at a reasonable price and integrate them together along with the two way mirror. 
  • Contingency plan is to have one smaller monitor that acts like a mirror and have the user stand further away so their entire image can still be seen. 
• Mirror UI: As mentioned in our presentation, another huge risk is how to display our Pose Correction results on the mirror itself and whether any further mapping may be required. We have spent some time working through the physics of it so it seems more plausible now but is still a risk until we acquire the hardware.
  • Contingency plan is to simply render the camera feed on monitor

Guiding Engineering/Scientific Principles For Our Design

• Machine Learning/Computer Vision:
  • Core components of our project like the pose estimation capabilities rely on computer vision ML models. The model we’ve selected to use is a convolutional neural network (CNN) model called BlazePose that is optimized for real-time, on-device inference
• Computer Graphics:
  • Need to use efficient and lightweight computer graphics libraries/methods in order to render pose corrective suggestions in real-time in a visually understandable way
• Software engineering:
  • Working with several python modules and writing a custom pose comparison library
• Geometry:
  • Pose comparison is based on angles and distances, which are rooted in 2D and 2D geometry
• Physics of light and mirrors:
  • Using the behavior of light and principles of reflection to understand where a person’s reflection will appear on the mirror
• Firmware and hardware-software interface:
  • Integrating camera input with the computer
  • Working with speakers for audio output
• Human-Computer Interaction (HCI):
  • Have HCI as an additional guiding principle to make sure that the final integrated system is easily usable and accessible for users.
  • Ensure that the UI and corrective feedback are delivered in a way that is understandable and doesn’t compromise the yoga learning experience (deliver audio cues to avoid user physically interacting with the screen for example)

Timeline Update:

For our software algorithm, we are on track. However, we were supposed to buy the hardware last week. We spoke to the TAs about using monitors that are already present in the lab and are waiting on that information to go ahead with the purchase of materials. We hope to make that purchase this week, which will push back our hardware timeline by a week.

Next Steps:

• Complete error comparison this week
• Order hardware and begin assembly of hardware
• Complete design presentation