Final Video
Corin’s Status Report for 12/6
Accomplishments
This week, we focused on finalizing the physical build of our smart mirror. We finally integrated all subsystems (camera, display, and mirror into the frame) and adjusted the setup to ensure the proper height and angle for the camera. This allowed a more consistent capture of the user at the intended operating distance mentioned in our design report.
We also worked on refining our user app based on testing and user study. One challenge was that even with a bigger display, it was really difficult to navigate the app with our fingers. We made all of the buttons/scroll on our app wider. In addition, we fixed lingering bugs in our positional feedback, adding accurate guidance based on the user location within the frame.
Schedule
Make our demo video & set up our project for final demo!!
Next Week
Work on the final report and demo video
Team Status Report for 12/6
Risks
After user testing, the most feedback we’ve gotten was the difficulty with the touchscreen interface. Even when the user pressed a button, our system’s app would not register it properly. To mitigate this, we made our essential buttons/controls larger so that it is easier to press.
We also had a lot of difficulty integrating the camera (it started throwing bugs after we switched the display). We fixed the problem by switching the camera port (for some reason, the original port threw errors for autofocus). We’re not sure why switching the port fixed the problem, but if we had to build the system again, these minor bugs are potential risks.
We also realized that during demo, where the mirror would sit affects user orientation. To make sure we don’t face difficulties on demo day, we will test it in the room we’ll have our demo in (somewhere in 1200 wing).
Changes
As we are approaching the end, we did not make any changes in our system expect for sizing of some buttons.
Schedule
No changes to the schedule! Our project has been finalized 🙂
Siena’s Status Report for 12/6
Accomplishments
Early this week, I used the new LCD (bigger size) and adjusted our system app’s UI so that everything is centered. Then, we conducted user study with 5 people in addition to all 3 of our group members (8 total).
After each user ran through 5 sessions, we received feedback about our UI as well as how helpful they thought the product was. We made changes accordingly, mostly with the UI (positional feedback and making buttons/scroll bars larger so its easier to press).
Schedule
We are on schedule! Waiting to set up our demo.
Next Week
Our team will work on the video and the final report.
Isaiah’s Status Report for 11/22
This week was finishing integration and some timing testing. The wrinkle model was integrated into the mirror code, and times were taken with all of the models together ~2.1 seconds for an image. In order to make this number smaller, I change the architecture of the wrinkle model to match that of the acne detection model, causing its individual inference time to go down to ~0.12 seconds. Beyond that, face detection was integrated with the mirror, although we had to switch from blazeface to DeepFace, due to RPi compatibility issues.
Team Status Report for 11/22
Risks and Mitigations
Our team has fully integrated all the software side of the system. The biggest risk for now is the physical connections not having stable connections for long term use. Since we changed around our physical design pretty late and put in the orders late as well, we have to speed up the process of CADing the new design and connecting the camera, display, lights, and the SBC in a stable manner.
Another risk is to have the user sessions be intuitive (touch screen display being comfortable to use), since we changed our design from using buttons to a touch screen. We designed the UI to have menu items + ordered a larger display, but we plan to conduct user study pretty soon to gather more information on how comfortable it is to use our product.
Our latency risk has been mostly fixed this week by parallelizing our models with threads and shortening the preview. The blaze face also does a great job quickly capturing our face for each session. We will continue testing the latency to ensure that we consistently hit the target latency.
Design Changes
We are using a real mirror and a 7 inch display so that the user can more clearly see their face and the analysis results. These were the physical changes mentioned last week (we have decided to confirm on the changes). Our overall system has the same functionality.
Schedule
Our team is pretty much on track! We plan to focus on testing next week.
Corin’s Status Report for 11/22
Accomplishments
This week, I worked with Siena to fully integrate the models we got from Isaiah with the app and finalized our UI. The wrinkle model got added to our app, and we confirmed that all four conditions were detected during a user session. We also tested the inference latency of our models and succeeded in parallelizing the 4 different models to reduce our latency. We achieved a latency of about 1.5 seconds, which was bottlenecked by our slowest model (the burn model). I created a trend graph with the last 5 sessions for the 4 different conditions. When the user clicks on the dots on the trend graph, it displays the photo result of that session. I also continued working on the new CAD sketch for our physical design. We changed the physical design from last week, and focused on seamlessly integrating a larger display so that the user can more comfortable navigate around the touch screen display.
Schedule
I am pretty much on track. I just need to speed up the CAD design, but I think it would be done by early next week.
Next Week
We hope to get a larger display, build the full system once again, then start collecting data from our tests.
Siena’s Status Report for 11/22
Accomplishments
This week, I worked on integrating the face detection model. After Isaiah has laid out the baseline, I fully integrated it fully so that it is compatible with the rest of the code. Our models yield more accurate results. I also worked on integrated the wrinkle model and adding our recommendations based on the top two most severe analysis.
Finally, we worked on threading. Our 4 ML models now don’t run sequentially, but take advantage of the 4 cores on our RPi. The following is the average result:
oil: 0.3083 s
acne: 0.4713 s
burn: 1.5153 s
wrinkle: 0.3870 s
—————————
total: 1.5167 s
It seems like the bottleneck is our burn model.
Schedule
Right now, we’re behind in terms of the physical build because we changed our design. We plan on finishing this early next week so we can start our user study the week after Thanksgiving.
Next Week
I hope we can put together the physical model before we leave for break. 
Corin’s Status Report for 11/15
Accomplishments
This week, I worked with Siena to modify our app. We added features to view the history of the user sessions (we added a view trend page that shows the past 5 analyses). We also added a mock recommendation system (based on the classification results, we display mock recommendations below each session results). We haven’t finalized the recommendation system yet because we’re still working on categorizing the different skin types based on the classification & confidence. We also added lighting to our product – this did make the input image better. Through testing, we also realized the distance to the camera also played significant role (the results were best when we were pretty close to the camera, less than 1ft).
I also planned out the new physical design we would implement if we were to change directions to a real mirror and a display underneath. A new CAD sketch is needed since the design would become slightly larger and a different frame is needed to accommodate the real mirror dimensions.
Schedule
I am on track!
Next Week
We hope to finalize our physical design and have the full physical product built. Siena and I will work on turning our mock recommendation system into a real one, and hopefully we will integrate all 4 models for skin analysis.