Team Status Report for 10/18

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Our current biggest risk is that the parts that we have ordered did not arrive. We have been stuck in this position for a while, so while the software has been making progress, our hardware aspect has not yet started yet. We checked in with our TA for our orders. In order to mitigate this situation, we have looked through a lot of resources and planned an even more thorough implementation for our hardware. Ideally, we’ll be able to bring up and connect them to the software and work as intended without much challenges in between. Additionally, another biggest risk is with our actual structure of the mirror. To account for our inexperience in this area, we have put in an order for a larger quantity of woodsheets so we have room for error. 

As for software, the biggest risk is still the performance of the remaining two models. The acne detection model is working well on the dataset. Likely, the largest risk is within the sunburn detection model. Increased data gain via scraping detailed in the report is likely the strongest risk management. 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

While making concrete plans for our design report, we have decided to add more buttons. This was necessary as our LCD will sit behind and mirror and will not have a touchable screen. These added buttons will help the user navigate our Magic MIrror App. Because buttons are cheap and are easily workable through GPIO, we don’t expect much added challenge. In the case where the user studies feedback details that buttons are hard to use, we may consider installing the LCD in front of the mirror (touchable screen). 

Provide an updated schedule if changes have occurred.

We changed our schedule on the implementation and testing side – our orders are coming in a bit late, and we realized that we need more time to integrate all the software/hardware components. We decided to shorten the user study by a week (we think this is reasonable since getting approved by the IRB will also take time).

link to our schedule 

Part A: written by Isaiah

Our project is designed around analyzing skin, and suggesting products to help improve skin health. There are multiple components to the project that require sensitivity to global considerations for the resulting product to be truly effective and accessible for a wide range of people. Beyond just skin tone, different environmental conditions such as average temperature and humidity of a region can shift. As an extreme case, the level of skin moisture that’s common and healthy in tropical regions might not be the same in mountainous or subarctic climates. Typical tips and tricks used to identify skin conditions that are popular in one climate or for one group of people might not be reliable for another. Our product allows for an algorithmic solution for analysing skin that’s trained and tested on a large variety of skin types, allowing for accurate analysis with a greater trend towards global invariance. Furthermore, in detailing generic products and product combinations, users can make use of the product recommendations, where specific brands might not be available in some regions of the world.

Part B: written by Corin

For any technology that requires personal data collection, many cultures, including the American culture, take privacy and trust very seriously. Privacy is often tied to personal rights, and there’s a cultural implication that personal information should be protected (laws for medical privacy, student records, etc. exemplify that). People generally expect transparency when it comes to how their information is collected, used, and shared. Mirror Mirror on the Wall focuses a lot on this privacy aspect, ensuring that all image processing is done on a raspberry pi locally. We intentionally made this design choice so that our user can trust that his/her data will be kept private and can comfortably use a product that collects sensitive personal data. 

Part C: written by Siena

Mirror Mirror on the Wall’s initiative is primarily focused on analysis of skin care and has no direct ties with environmental factors. However, we have considered environmental impacts in our design to offer sustainable use of resources. Through the use of a Raspberry Pi 5, an energy-efficient embedded system, and making all machine learning inference locally inside the device instead of relying on cloud servers, our system circumvents network-based carbon emissions. Although the physical components of the mirror (LCD, camera, case, etc) must have electronic materials as part of them, our design encourages durability, modularity, and reuse, such that individual components are repairable or replaceable without scrapping the entire system. While the system never has a direct interaction with natural ecosystems or biological systems, its impact on the environment is reduced indirectly through employing sustainable hardware options and power conservation. 

 

Corin’s Status Report for 10/18

What did you personally accomplish this week on the project?

This week, we had to focus a lot on the design report. I wrote the abstract, introduction, user-case requirements, design requirements, and the hardware side of the design trade studies. I realized that we didn’t include design trade studies in our design presentation, and included tables in the design report to visualize our reasonings behind our component choices. 

For implementation, this week has been the slowest. I am still waiting on my LCD display to connect it to the RPI. Although I did some research on the app and recommender system last week, I realized that there needs to be more planning on the software side to really integrate the button -> camera -> ml model -> recommender system -> app. Therefore, I changed directions to set up the Raspberry pi and learn the different libraries for an easier integration.

Is your progress on schedule or behind?

My progress is behind. I am planning to ramp up the pace after break. The software integration needs more planning, and I will discuss with Siena and Isaiah after break to combine the software sections.

What deliverables do you hope to complete in the next week?

While planning and gradually combining the software side, I’m hoping that the rpi will be set up with basic components connected (button/display). Hopefully I can test the basic connections with a mock app or just python code. 



Corin’s Status Report for 10/4

What did you personally accomplish this week on the project?

This week, our team put in the orders for most of our design parts.

In terms of personal accomplishment, the progress was a bit slower because we took longer to put in our order. Instead of focusing on connecting the physical components, I started looking into constructing the recommender system and the on system app for the user interface. With siena’s list of skin care products and the outputted classification/confidence from Isaiah’s ml work, I started working on the recommender system.

I also looked into creating our on system app to display on the mirror. Both the recommender system and the on system app are in progress, but I hope to be able to have a mock display by next week.

I also thought it would be best to modify our CAD next week when we do get all our components and finalize our physical design.

Finally, we also spent time drafting our design report and working on the diagrams that will be added to our report.

Is your progress on schedule or behind?

My progress is slightly behind because we didn’t receive the parts except the camera and the board. However, because we are using a touch screen instead of the SPI LCD, we don’t need to spend much time on the communication between the RPI and the LCD but need to focus on creating the on-system app. This can be done without the LCD. I wouldn’t say we’re too behind, we just changed our schedule to focus on a different side of the project.

What deliverables do you hope to complete in the next week?

I would like to have code running for the recommendation system and to see if I can create a small mock app for our LCD. 

Corin’s Status Report for 9/27

What did you personally accomplish this week on the project?

This week, I mainly worked with the team to complete our overarching design. In the beginning of the week, all of us decided on using a RPI 5 as our main SBC board and realized that we wouldn’t need additional microcontrollers or accelerators for now. We decided that the RPI camera module 3 and the RPI 5 inch touch screen display would be best compatible with our RPI while satisfying the necessary requirements (size/resolution). Both Siena and I worked on planning out how all the different hardware components will be laid out. I also created a CAD model of our physical design, planning out the dimensions of the actual mirror (12inch * 8inch) and placing the camera, touch screen display, and RPI in positions such that all necessary connections can be made.

Is your progress on schedule or behind?

My progress is on schedule. Last week, I mentioned that I want to focus on the LCD display because I thought we might be using an LCD display connected via GPIO + a communication protocol like i2c. Since our group decided on a touch screen display connected via HDMI (micro HDMI), we don’t need to worry about the pinout or a separate communication protocol. However, we will have to think about launching an app on our RPI that will be displayed on our screen.

What deliverables do you hope to complete in the next week?

For next week, I would like to modify the CAD design to a more detailed level after receiving all of our parts. I would also like to connect the camera and LCD display to the RPI and test the input/output of the images to the processor and processor data to LCD display (although the latter I think might take more than a week to set up the app).


Corin’s Status Report for 9/20

What did you personally accomplish this week on the project? 

This week, I wanted to find a board that best suited our project. While Siena looked into raspberry pi boards, I looked into the Nvidia Jetson Orin Nano Super Developer Kit.

Below are the components of the board.

Price ($250)

I researched the basic setup that needs to be done when we get out board.

Hardware board setup: https://developer.nvidia.com/embedded/learn/get-started-jetson-orin-nano-devkit#intro

Software setup : Nvidia JetPack (software stack) – including Tesnsor RT, pytorch, etc

Choose to build project from source:

https://github.com/dusty-nv/jetson-inference/blob/master/docs/building-repo-2.md

After the steps above, pytorch will be installed, and either python or c++ can be used.

Below is a link to start coding our own image recognition program

https://github.com/dusty-nv/jetson-inference/blob/master/docs/imagenet-example-python-2.md

 

For a live camera demo, 

An MIPI CSI Camera can be used (port 9 of the board image above).

Raspberry Pi Camera Module 2 ($12)

AP-IMX290-MIPIYUVx1 – harder to buy

AP-IMX334-MIPIx1 – harder to buy

We can also use USB webcams  (port 6 of our board image above).

Logitec C270($22)

Logitec C920($60) 

 

For LCD, we can use 

7 Inch IPS LCD Touch Screen Display ($46) 

5 Inch IPS LCD Touch Screen Display ($40)

The one concern is that there aren’t many resources out there on LCD displays for Nvidia Jetson orin nano boards. I’m pretty sure the two displays above will work with the cable from DP-HDMI (jetson orin nano does not have an HDMI port, only a DP).

Another option is to use TFT LCDs and use the gpio pins on the board. This will be a cheap option for us, since the costs will be below $30, but the display will be small.

More research has to be done on how to communicate the output data to our LCD. Communication methods will vary based on the LCD we decide on.

 

Is your progress on schedule or behind?

My progress is pretty much on schedule, but I want to focus more on the LCD aspect next week. I want to decide which display our project will use (will it be connected via DP or GPIOs), and exactly how the software side should progress to deliver our output data to the display.

 

What deliverables do you hope to complete in the next week?

I hope to have chosen our board, and to have a plan on how the output data will be delivered to our LCD. I don’t think we’ll have our board by next week, but based on our board decision, I plan to look at the datasheets for hardware configuration of the LCD and research how the communication will work from the board to the display.