Posted on

Isaiah’s Status Report for 10/18

What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficienteffort into the project over the course of the week (12+ hours)

A large amount of time this week was spent with drafting the Design Report. Besides that, a majority of the time remaining was spent training and developing the machine learning models for the skincare analysis. In particular the acne detection model took a large amount of the time. Developing the code for the YOLO used in the acne detection was a more difficult task than I initially thought, although the end results were quite satisfactory.  I took guidance from the official YOLO repository, which helped make the process smoother

Here’s a mosaic of detections from the validation set

Since the model is a detection model, I don’t yet have a classification accuracy value yet, but visually the detections look good. Classification would be completed via checking for the presence of acne within a patch, and if it’s there (and with a strong enough confidence), then classifying the patch as having acne. This also gives the ability to have a richer value than just a binary output for acne classification.

The wrinkle detection model is in progress leading likely the sunburn model to be the last one completed.

Posted on

Team Status Report for 10/18

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Our current biggest risk is that the parts that we have ordered did not arrive. We have been stuck in this position for a while, so while the software has been making progress, our hardware aspect has not yet started yet. We checked in with our TA for our orders. In order to mitigate this situation, we have looked through a lot of resources and planned an even more thorough implementation for our hardware. Ideally, we’ll be able to bring up and connect them to the software and work as intended without much challenges in between. Additionally, another biggest risk is with our actual structure of the mirror. To account for our inexperience in this area, we have put in an order for a larger quantity of woodsheets so we have room for error. 

As for software, the biggest risk is still the performance of the remaining two models. The acne detection model is working well on the dataset. Likely, the largest risk is within the sunburn detection model. Increased data gain via scraping detailed in the report is likely the strongest risk management. 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

While making concrete plans for our design report, we have decided to add more buttons. This was necessary as our LCD will sit behind and mirror and will not have a touchable screen. These added buttons will help the user navigate our Magic MIrror App. Because buttons are cheap and are easily workable through GPIO, we don’t expect much added challenge. In the case where the user studies feedback details that buttons are hard to use, we may consider installing the LCD in front of the mirror (touchable screen). 

Provide an updated schedule if changes have occurred.

We changed our schedule on the implementation and testing side – our orders are coming in a bit late, and we realized that we need more time to integrate all the software/hardware components. We decided to shorten the user study by a week (we think this is reasonable since getting approved by the IRB will also take time).

link to our schedule 

Part A: written by Isaiah

Our project is designed around analyzing skin, and suggesting products to help improve skin health. There are multiple components to the project that require sensitivity to global considerations for the resulting product to be truly effective and accessible for a wide range of people. Beyond just skin tone, different environmental conditions such as average temperature and humidity of a region can shift. As an extreme case, the level of skin moisture that’s common and healthy in tropical regions might not be the same in mountainous or subarctic climates. Typical tips and tricks used to identify skin conditions that are popular in one climate or for one group of people might not be reliable for another. Our product allows for an algorithmic solution for analysing skin that’s trained and tested on a large variety of skin types, allowing for accurate analysis with a greater trend towards global invariance. Furthermore, in detailing generic products and product combinations, users can make use of the product recommendations, where specific brands might not be available in some regions of the world.

Part B: written by Corin

For any technology that requires personal data collection, many cultures, including the American culture, take privacy and trust very seriously. Privacy is often tied to personal rights, and there’s a cultural implication that personal information should be protected (laws for medical privacy, student records, etc. exemplify that). People generally expect transparency when it comes to how their information is collected, used, and shared. Mirror Mirror on the Wall focuses a lot on this privacy aspect, ensuring that all image processing is done on a raspberry pi locally. We intentionally made this design choice so that our user can trust that his/her data will be kept private and can comfortably use a product that collects sensitive personal data. 

Part C: written by Siena

Mirror Mirror on the Wall’s initiative is primarily focused on analysis of skin care and has no direct ties with environmental factors. However, we have considered environmental impacts in our design to offer sustainable use of resources. Through the use of a Raspberry Pi 5, an energy-efficient embedded system, and making all machine learning inference locally inside the device instead of relying on cloud servers, our system circumvents network-based carbon emissions. Although the physical components of the mirror (LCD, camera, case, etc) must have electronic materials as part of them, our design encourages durability, modularity, and reuse, such that individual components are repairable or replaceable without scrapping the entire system. While the system never has a direct interaction with natural ecosystems or biological systems, its impact on the environment is reduced indirectly through employing sustainable hardware options and power conservation. 

 

Posted on

Corin’s Status Report for 10/18

What did you personally accomplish this week on the project?

This week, we had to focus a lot on the design report. I wrote the abstract, introduction, user-case requirements, design requirements, and the hardware side of the design trade studies. I realized that we didn’t include design trade studies in our design presentation, and included tables in the design report to visualize our reasonings behind our component choices. 

For implementation, this week has been the slowest. I am still waiting on my LCD display to connect it to the RPI. Although I did some research on the app and recommender system last week, I realized that there needs to be more planning on the software side to really integrate the button -> camera -> ml model -> recommender system -> app. Therefore, I changed directions to set up the Raspberry pi and learn the different libraries for an easier integration.

Is your progress on schedule or behind?

My progress is behind. I am planning to ramp up the pace after break. The software integration needs more planning, and I will discuss with Siena and Isaiah after break to combine the software sections.

What deliverables do you hope to complete in the next week?

While planning and gradually combining the software side, I’m hoping that the rpi will be set up with basic components connected (button/display). Hopefully I can test the basic connections with a mock app or just python code. 



Posted on

Siena’s Status Report for 10/18

Accomplishments
At the start of the week, our team divided the design report into sections. My part was specifically on system architecture, implementation, project management, risk mitigation, and summary (please refer to the corresponding sections of our design report). While we had big ideas in place, I realized that we needed more concrete plans when actually writing the report. I thus refined our hardware architecture/implementation diagrams, defined how our button controls were going to work, and detailed the hardware-software as well as peripheral interface. In addition, by following the CAD model, I added more products to our parts list.

Our parts didn’t arrive yet, but I started working with the RPi camera module for now. I discussed with Isaiah on the methods we want to use to pass the frames to the software side (function call, direct software management, etc).

I also reached out to the IRB office, explaining our plans for our user study and what processes we need to go through in order to get it approved. The email is included below.

Schedule
Unfortunately, we are currently behind schedule. Our parts are not here, and this has been our bottleneck since the last week. We are discussing plans to cut down the time reserved for our user study from 2 weeks to 1. This way, we’ll have another week to actually work on our product.

Deliverables
Hopefully by next week, our parts would be here. My goal would be to have the camera working and feeding in correctly to our Magic Mirror app.

Posted on

Siena’s Status Report for 10/4

Accomplishments

This week, I worked on building our parts list from our discussion based off of our design review slides from last week. Our group had already ordered a RPi 5 from the inventory, so we ordered additional components like our camera, LCD, LEDs, and more. Aside from parts, I was messing around with the RPi we picked up. Although there wasn’t much that I could do without the rest of our parts, I did extensive research as to how we will integrate the camera and buttons. Here are the main resources for setup, and I have also created an initial code based off of them to take a picture when a button is pressed. It has not been tested yet, but I will do that when everything gets here. 

Parts Status:

Resources for buttons for control: https://gpiozero.readthedocs.io/en/stable/recipes.html#button

Resources for camera setup: https://docs.arducam.com/Raspberry-Pi-Camera/Native-camera/16MP-IMX519/#products-list 

Schedule

For week 6, we are on schedule. We have ordered all the parts that we need in order to proceed with our projects, and I have looked into different ways that I can bring up the camera and button, as well as how they communicate with each other and integrate through the RPi.

Next Week

In the next week, I think our group’s greatest concern is finishing up our design review documents. On a personal level, I wish to be able to bring up the camera and control it with buttons when the parts get here.



Posted on

Team Status Report for 10/4

  • What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risks haven’t changed since last week: it is still making sure our user design is intuitive, and our models are accurate.  We developed our physical design a bit more this week, and we discussed what the general flow of the Magic Mirror app should be to ensure ease of use.

In terms of hardware risk, the integration between the new IMX519 camera and the Raspberry Pi 5 introduces potential driver and compatibility challenges. To manage this, we’ve already researched and documented how to bring up the camera using the new picamera2 and libcamera stack. If hardware communication issues arise, our fallback plan is to test using a standard Pi camera first to validate the GPIO and control flow before reintroducing the IMX519. This ensures we can continue development on schedule even if certain hardware components take longer to configure.

  • Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward

One of the only significant changes was adding more buttons to the system for better system control/user experience. We initially decided on having only one button, but as we planned out the app, we realized that navigating it with only one button would be confusing and inconvenient. We have many open GPIO pins on the RPi, and the buttons are still cheap, so this would mainly just affect our physical structure design.

  • Provide an updated schedule if changes have occurred.

No changes to the schedule as of now.

Posted on

Isaiah’s Weekly Status Report for 10/4

What did you personally accomplish this week on the project? Give files orphotos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week

I went back and updated the oily model as described in last week’s report, and I had found successes. The model had learned and performs well on the test dataset. The model achieved at best 87% test accuracy, which is satisfactory. Also, the validation was done with noise added, so the true accuracy might be slightly higher. As such, I moved on to the acne detection. At first, I tried a similar method, but didn’t find much success. I then decided to do more research, and found that both my dataset, and many existing methods do acne detection (generating bounding boxes for acne). So, I decided to start developing an acne detection model, and just the presence or absence of acne to classify any given face. That is in the works as I write this.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?What deliverables do you hope to complete in the next week?

Progress is on schedule, I have 1 model done, likely the hardest, and another model that will be done soon. This leaves next week and fall break for training the last two models. Wrinkles will likely be tackled in a similar manor to acne as a detection problem.

Posted on

Corin’s Status Report for 10/4

What did you personally accomplish this week on the project?

This week, our team put in the orders for most of our design parts.

In terms of personal accomplishment, the progress was a bit slower because we took longer to put in our order. Instead of focusing on connecting the physical components, I started looking into constructing the recommender system and the on system app for the user interface. With siena’s list of skin care products and the outputted classification/confidence from Isaiah’s ml work, I started working on the recommender system.

I also looked into creating our on system app to display on the mirror. Both the recommender system and the on system app are in progress, but I hope to be able to have a mock display by next week.

I also thought it would be best to modify our CAD next week when we do get all our components and finalize our physical design.

Finally, we also spent time drafting our design report and working on the diagrams that will be added to our report.

Is your progress on schedule or behind?

My progress is slightly behind because we didn’t receive the parts except the camera and the board. However, because we are using a touch screen instead of the SPI LCD, we don’t need to spend much time on the communication between the RPI and the LCD but need to focus on creating the on-system app. This can be done without the LCD. I wouldn’t say we’re too behind, we just changed our schedule to focus on a different side of the project.

What deliverables do you hope to complete in the next week?

I would like to have code running for the recommendation system and to see if I can create a small mock app for our LCD. 

Posted on

Corin’s Status Report for 9/27

What did you personally accomplish this week on the project?

This week, I mainly worked with the team to complete our overarching design. In the beginning of the week, all of us decided on using a RPI 5 as our main SBC board and realized that we wouldn’t need additional microcontrollers or accelerators for now. We decided that the RPI camera module 3 and the RPI 5 inch touch screen display would be best compatible with our RPI while satisfying the necessary requirements (size/resolution). Both Siena and I worked on planning out how all the different hardware components will be laid out. I also created a CAD model of our physical design, planning out the dimensions of the actual mirror (12inch * 8inch) and placing the camera, touch screen display, and RPI in positions such that all necessary connections can be made.

Is your progress on schedule or behind?

My progress is on schedule. Last week, I mentioned that I want to focus on the LCD display because I thought we might be using an LCD display connected via GPIO + a communication protocol like i2c. Since our group decided on a touch screen display connected via HDMI (micro HDMI), we don’t need to worry about the pinout or a separate communication protocol. However, we will have to think about launching an app on our RPI that will be displayed on our screen.

What deliverables do you hope to complete in the next week?

For next week, I would like to modify the CAD design to a more detailed level after receiving all of our parts. I would also like to connect the camera and LCD display to the RPI and test the input/output of the images to the processor and processor data to LCD display (although the latter I think might take more than a week to set up the app).


Posted on

Siena’s Status Report for 9/27

Accomplishments

This week, our group held extensive discussions to solidify our implementation plans, ranging from component selection to defining high-level project goals. I was primarily responsible for the hardware design, focusing on how our chosen single-board computer would serve as the central hub of the magic mirror. This included planning hardware interfaces and ensuring it could host both our system application and the ML component.

Working closely with Isaiah, who is leading the software side, I finalized a condensed design plan after several iterations. We revisited our design goals multiple times, aligning on the user experience we wanted to deliver. While creating a detailed diagram, I also researched relevant resources and libraries that could support our implementation. After several drafts, we arrived at a finalized version of the design plan:

Schedule

I am currently on track. This week’s goal was to finalize our parts list and research how the board would interface with components such as the control buttons and camera, which has been completed.

Next Week

Next week, we expect to receive some of the parts we ordered. Since we will at least have access to the Raspberry Pi and camera from the capstone inventory, I plan to begin testing simple connections with the camera. This will help me gauge if my interface plans were sufficient in regards to the camera and decide on next steps.