Category: Siena’s Status Reports

Accomplishments

This week, I worked on integrating the face detection model. After Isaiah has laid out the baseline, I fully integrated it fully so that it is compatible with the rest of the code. Our models yield more accurate results. I also worked on integrated the wrinkle model and adding our recommendations based on the top two most severe analysis. 

Finally, we worked on threading. Our 4 ML models now don’t run sequentially, but take advantage of the 4 cores on our RPi. The following is the average result:

oil: 0.3083 s

acne: 0.4713 s

burn: 1.5153 s

wrinkle: 0.3870 s

—————————

total: 1.5167 s

It seems like the bottleneck is our burn model. 

Schedule
Right now, we’re behind in terms of the physical build because we changed our design. We plan on finishing this early next week so we can start our user study the week after Thanksgiving. 

Next Week

I hope we can put together the physical model before we leave for break. 

Accomplishments

This week, Corin and I worked on further developing our system app. First, we clearly defined what we want to output to the users for each of the four categories we have. This analysis output is now stored, and up to 5 of the most recent results are showed on our new “View History” page. 

Corin and I also added LEDs to our interim setup. After testing it, we found that it gave more accurate results like expected. 

Last but not least, I was involved in discussions about altering our physical setup after the feedback our team got after the demo. 

Schedule

Mostly on track, a little bit behind because we may need to do another iterations of our physical design.

Next Week

I hope our team can finish our iteration after feedback and make a refined physical model. I also want to work on actually adding the recommendation system. 

Accomplishments

This week, we actually began integration. Our group members worked together most of the time, but I concentrated on connecting the camera inputs to the system’s app, showing the preview of the camera before taking the picture, saving the picture, and passing the picture taken to the ML models that Isaiah has made. There were a few hiccups in the environment setup and actually making it support both the camera and the ML model.  However, all were resolved. 

In addition, Corin and I laser cut the plywoods and made a mock setup for our interim demo. The both of us also refined the system app together for a more pleasant user interface. 

Schedule

We are currently on schedule!

Next Week

I hope to now integrate the recommender system, as now we are able to extract and display the outputs from our analysis. 

Accomplishments

Mid-week, we finally received our parts and could begin working on our project. However, I had a lot of problem setting up the new camera. I tried following the guidance on the website, watching youtube tutorials, reading through documentation, and more. However, nothing seemed to work. I spent most of the week debugging, but couldn’t get it working. I’m planning on asking the TAs on Monday.

Because I couldn’t make much progress with the camera, I helped Corin with bringing up the LCD. I mostly worked on the connection while Corin was working on our system app. Whilst working, we took a short video showing how the LCD would show through the two-way mirror. The “two-way” of the mirror worked well, but we are a little concerned with the “mirror” functionality as the color is a little bit distorted. 

     

Schedule

We are currently on schedule. 

Next Week

I hope to get the camera working early next week. Afterwards, I want to work on the actual integration with the ML model that Isaiah has been working on. I expect to be able to have some kind of analysis result by the end of next week.

Additionally, I hope to talk to one of the advisors to move forward in the user study approval process.

Accomplishments

Unfortunately, our parts are still not here 🙁 I’ve been trying to find ways around this, and I have started working with a different camera (Camera Module 3). Although the process and the API might be a little different, I was still able to bring up the camera. Through this, our team was able to discuss ways how we will actually interface with our system’s app. We are still trying out different methodologies to find one that works best for us. 

I also put in orders for now our exterior of our project (wood and two-way mirror glass) and other tools we need to put it together. 

Last but not least, I’ve been communicating with the IRB Review. After explaining our capstone project and the purpose of our user study, they have sent us a checklist to work through with our advisor. I have worked through the checklist and am now waiting for our next week’s meeting where we can get the advisor approval. 

Course-Related Student Project Checklist – 1-4-22 – FINAL

Schedule

After we have altered the schedule last week, I think we are currently on track. 

Deliverables

Next week, I hope we can run our ML model with the inputs from the camera (it’ll be even better if we can work on this with the camera that we ordered, which has higher resolution), have it interface with memory, and collect some results on the analysis. 

Accomplishments

This week, I worked on building our parts list from our discussion based off of our design review slides from last week. Our group had already ordered a RPi 5 from the inventory, so we ordered additional components like our camera, LCD, LEDs, and more. Aside from parts, I was messing around with the RPi we picked up. Although there wasn’t much that I could do without the rest of our parts, I did extensive research as to how we will integrate the camera and buttons. Here are the main resources for setup, and I have also created an initial code based off of them to take a picture when a button is pressed. It has not been tested yet, but I will do that when everything gets here. 

Parts Status:

Resources for buttons for control: https://gpiozero.readthedocs.io/en/stable/recipes.html#button

Resources for camera setup: https://docs.arducam.com/Raspberry-Pi-Camera/Native-camera/16MP-IMX519/#products-list 

Schedule

For week 6, we are on schedule. We have ordered all the parts that we need in order to proceed with our projects, and I have looked into different ways that I can bring up the camera and button, as well as how they communicate with each other and integrate through the RPi.

Next Week

In the next week, I think our group’s greatest concern is finishing up our design review documents. On a personal level, I wish to be able to bring up the camera and control it with buttons when the parts get here.

Accomplishments

This week, our group held extensive discussions to solidify our implementation plans, ranging from component selection to defining high-level project goals. I was primarily responsible for the hardware design, focusing on how our chosen single-board computer would serve as the central hub of the magic mirror. This included planning hardware interfaces and ensuring it could host both our system application and the ML component.

Working closely with Isaiah, who is leading the software side, I finalized a condensed design plan after several iterations. We revisited our design goals multiple times, aligning on the user experience we wanted to deliver. While creating a detailed diagram, I also researched relevant resources and libraries that could support our implementation. After several drafts, we arrived at a finalized version of the design plan:

Schedule

I am currently on track. This week’s goal was to finalize our parts list and research how the board would interface with components such as the control buttons and camera, which has been completed.

Next Week

Next week, we expect to receive some of the parts we ordered. Since we will at least have access to the Raspberry Pi and camera from the capstone inventory, I plan to begin testing simple connections with the camera. This will help me gauge if my interface plans were sufficient in regards to the camera and decide on next steps. 

Accomplishments

This week, I researched the hardware components for our project such as the single-board computer (SBC), camera, and LCD. My main responsibility was evaluating the Raspberry Pi as a potential system controller. During this process, I realized that the two microcontrollers (MCUs) in our original design were unnecessary, since I found no trouble in connecting both the camera and LCD directly to the SBC. I also identified an AI accelerator plug-in to better custom the Raspberry Pi with our needs. Meanwhile, one of my teammates researched the NVIDIA Jetson, and we plan to discuss both options with our advisors next week.

In addition, I researched treatment approaches for the four skin conditions we will analyze (acne, oiliness, wrinkles, and sunburn) by reviewing online resources and recommendations.

My work is here Week 4

Schedule

We are slightly behind schedule because our group wanted guidance on selecting the SBC and needed approval for our revised hardware plan (which removes the two MCUs). To ensure we can catch up quickly, my groupmate and I have researched both SBC options under consideration, along with the corresponding camera and LCD for each. With this groundwork completed, we expect to be back on track by early next week.

Next Week

Next week, we aim to finalize our parts list, get the updated diagram approved, and begin working on the communication protocols for the selected components. I plan to compile a list of relevant SDKs and draft an initial plan for how to implement them.