Posted on

Corin’s Status Report for 12/6

Accomplishments

This week, we focused on finalizing the physical build of our smart mirror. We finally integrated all subsystems (camera, display, and mirror into the frame) and adjusted the setup to ensure the proper height and angle for the camera. This allowed a more consistent capture of the user at the intended operating distance mentioned in our design report.

We also worked on refining our user app based on testing and user study. One challenge was that even with a bigger display, it was really difficult to navigate the app with our fingers. We made all of the buttons/scroll on our app wider. In addition, we fixed lingering bugs in our positional feedback, adding accurate guidance based on the user location within the frame.

Schedule

Make our demo video & set up our project for final demo!!

Next Week

Work on the final report and demo video

Posted on

Team Status Report for 11/22

Risks and Mitigations

Our team has fully integrated all the software side of the system. The biggest risk for now is the physical connections not having stable connections for long term use. Since we changed around our physical design pretty late and put in the orders late as well, we have to speed up the process of CADing the new design and connecting the camera, display, lights, and the SBC in a stable manner.

Another risk is to have the user sessions be intuitive  (touch screen display being comfortable to use), since we changed our design from using buttons to a touch screen.  We designed the UI to have menu items + ordered a larger display, but we plan to conduct user study pretty soon to gather more information on how comfortable it is to use our product.

Our latency risk has been mostly fixed this week by parallelizing our models with threads and shortening the preview. The blaze face also does a great job quickly capturing our face for each session. We will continue testing the latency to ensure that we consistently hit the target latency.

Design Changes

We are using a real mirror and a 7 inch display so that the user can more clearly see their face and the analysis results. These were the physical changes mentioned last week (we have decided to confirm on the changes).  Our overall system has the same functionality.

Schedule

Our team is pretty much on track! We plan to focus on testing next week.

Posted on

Corin’s Status Report for 11/22

Accomplishments

This week, I worked with Siena to fully integrate the models we got from Isaiah with the app and finalized our UI. The wrinkle model got added to our app, and we confirmed that all four conditions were detected during a user session. We also tested the inference latency of our models and succeeded in parallelizing the 4 different models to reduce our latency. We achieved a latency of about 1.5 seconds, which was bottlenecked by our slowest model (the burn model). I created a trend graph with the last 5 sessions for the 4 different conditions. When the user clicks on the dots  on the trend graph, it displays the photo result of that session.  I also continued working on the new CAD sketch for our physical design. We changed the physical design from last week, and focused on seamlessly integrating a larger display so that the user can more comfortable navigate around the touch screen display.

Schedule

I am pretty much on track. I just need to speed up the CAD design, but I think it would be done by early next week.

Next Week

We hope to get a larger display, build the full system once again, then start collecting data from our tests.

 

 

 

Posted on

Siena’s Status Report for 11/22

Accomplishments

This week, I worked on integrating the face detection model. After Isaiah has laid out the baseline, I fully integrated it fully so that it is compatible with the rest of the code. Our models yield more accurate results. I also worked on integrated the wrinkle model and adding our recommendations based on the top two most severe analysis. 

Finally, we worked on threading. Our 4 ML models now don’t run sequentially, but take advantage of the 4 cores on our RPi. The following is the average result:

oil: 0.3083 s

acne: 0.4713 s

burn: 1.5153 s

wrinkle: 0.3870 s

—————————

total: 1.5167 s

It seems like the bottleneck is our burn model. 

Schedule
Right now, we’re behind in terms of the physical build because we changed our design. We plan on finishing this early next week so we can start our user study the week after Thanksgiving. 

Next Week

I hope we can put together the physical model before we leave for break. 

Posted on

Corin’s Status Report for 11/15

Accomplishments

This week, I worked with Siena to modify our app. We added features to view the history of the user sessions (we added a view trend page that shows the past 5 analyses). We also added a mock recommendation system (based on the classification results, we display mock recommendations below each session results). We haven’t finalized the recommendation system yet because we’re still working on categorizing the different skin types based on the classification & confidence. We also added lighting to our product – this did make the input image better. Through testing, we also realized the distance to the camera also played significant role (the results were best when we were pretty close to the camera, less than 1ft).

I also planned out the new physical design we would implement if we were to change directions to a real mirror and a display underneath. A new CAD sketch is needed since the design would become slightly larger and a different frame is needed to accommodate the real mirror dimensions.

Schedule

I am on track!

Next Week

We hope to finalize our physical design and have the full physical product built. Siena and I will work on turning our mock recommendation system into a real one, and hopefully we will integrate all 4 models for skin analysis.

Posted on

Team Status Report for 11/15

Risks and Mitigations

This week, after the interim demo, we got some feedback on improving the physical design. It was harder to see through the mirror film for our screen display than we imagined. One suggestion made to our group was to change our design so that we have an actual mirror and the display right beneath the actual mirror instead of a see through mirror film. This would improve the quality of the reflection as well as the screen display. Although we are still contemplating on what design to go with, we have placed and received the parts to change our design as suggested.

We also got feedback on our latency for our inference models. We didn’t have the time measured for the inference models, and we got questions on whether our product will meet the timing we aimed for. After the interim demo, Isaiah measured the time for each inference models, which averaged 2.01 seconds. We are planning to figure out a way to run the models in parallel to achieve the timing for our MVP.

Besides the two concerns mentioned above, we are still working on improving the accuracy of our models. Isaiah is still working with the models to improve the accuracy, while Siena and Corin are working with the lighting that is optimal for the input image.

Design Changes

As mentioned in the first part of the section above, we are contemplating on getting rid of the mirror film and replacing it with an actual mirror. The display will be beneath the mirror instead of behind the mirror film, if the design changes. None of the software side will change, and our product will still deliver the same functionality (just a slight modification in the physical side of the product).

Schedule

We are pretty much on track!

 

Posted on

Siena’s Status Report for 11/15

Accomplishments

This week, Corin and I worked on further developing our system app. First, we clearly defined what we want to output to the users for each of the four categories we have. This analysis output is now stored, and up to 5 of the most recent results are showed on our new “View History” page. 

Corin and I also added LEDs to our interim setup. After testing it, we found that it gave more accurate results like expected. 

Last but not least, I was involved in discussions about altering our physical setup after the feedback our team got after the demo. 

Schedule

Mostly on track, a little bit behind because we may need to do another iterations of our physical design.

Next Week

I hope our team can finish our iteration after feedback and make a refined physical model. I also want to work on actually adding the recommendation system. 

Posted on

Corin’s Status Report for 11/8

Accomplishments

This week, our team started integrating all of our subsystems together. My job mainly was to assist Siena and Isaiah integrate the code they had for the camera + ml model to the mock app that I have previously built. I also quickly did a little bit of CAD work to laser cut the mirror frame. The wood we got was a bit smaller than we expected it to be, so I made a smaller version of the CAD sketches to make a prototype for our interim demo. I also added extra features to the app that we had before – Isaiah added code to outline the conditions on the image, and I added the feature on the app so that the user can view their picture with analysis on it.

Schedule

I am on schedule now. The next steps for me would be to modify the CAD so that we have a product that is well put together. I would also like to improve on the user experience (refining the app and the buttons).

Deliverables

Currently, we have a prototype with acne, sunburn, and oiliness analysis. Our next step is to add wrinkles as well and to improve the overall quality of our product with a better CAD design and lighting.

Posted on

Team Status Report for 11/8

Risks and Mitigations

This week, we worked on integration and put together a mock mirror for our interim demo. Through this process and by testing the integrated model, we noticed a few challenges. 

  • Reflection – The quality of our “mirror” functionality is low. We plan to address this by adding a black background directly behind the two way mirror (in areas excluding our LCD, of course).
  • Lighting – Lighting heavily impacts our analysis results. We have yet to install the LED lights discussion in our design report, so adding that will help us get more accurate results. 
  • Rounded cornet design – The plywoods we currently have don’t bend well, making it difficult for us to fill the edges of our casing. We plan to look into a different material for this. However, it would also be fine to have 90 degree corners instead of the rounded ones as it will just affect the appearance of the mirror, not the functionality.

 

Design Changes

After testing the mirror, we decided to add a preview feature before a picture is taken so the user gets more feedback from the system. It reduces confusion on the user’s end as they don’t have to guess about whether they are in frame or not. However, this won’t affect other aspects of our design. 

Schedule

No change to schedule! We are on track 🙂 

Posted on

Siena’s Status Report for 11/8

Accomplishments

This week, we actually began integration. Our group members worked together most of the time, but I concentrated on connecting the camera inputs to the system’s app, showing the preview of the camera before taking the picture, saving the picture, and passing the picture taken to the ML models that Isaiah has made. There were a few hiccups in the environment setup and actually making it support both the camera and the ML model.  However, all were resolved. 

In addition, Corin and I laser cut the plywoods and made a mock setup for our interim demo. The both of us also refined the system app together for a more pleasant user interface. 

Schedule

We are currently on schedule!

Next Week

I hope to now integrate the recommender system, as now we are able to extract and display the outputs from our analysis.