Author: aazhao

Alanis’ Status Report for 10/19

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours.

Over the past two weeks, I made a large pivot in the way that I addressed model architecture and training. Our model was insufficiently accurate and I believe it was due to three things. First, I tried to train one ResNet50 model to classify three things (clothing type, color, and usage), which was difficult to get high accuracy on because this is a very complicated task. Furthermore, I treated the task as multilabel classification, where each of the labels(each clothing type, color, and usage) is either true or false. This is partially incorrect because each item only belongs to one clothing type, base color, and usage category. Also, I was adding additional layers onto the pretrained ResNet50, which was harming the training because I was adding layers of random weights to pre-calibrated weights of the ResNet50. I decided to pivot to having 3 individual ResNet50 models classify for each category. Based on our research, which is discussed in more detail in our design report, having 3 ResNet50 models is within our range for total model size and classification time. I then saved each model onto my computer so I can later upload them to the Jetson. 

I then changed my code to train 3 different models, which is located on our Github. I then downloaded the frontend repo, which is located here, and began familiarizing myself with the structure to start helping Gabriella with the frontend. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the proiect schedule?

My progress is on schedule.

What deliverables do you hope to complete in the next week?

I hope to help Riley figure out using the OpenCV library on the Jetson Nano so we can start integrating our models in. I also hope to help Gabriella make progress on the frontend.

Alanis’ Status Report for 10/5

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours.

 I tried two different approaches to training a model. The first was to use a four-layer convolutional neural network architecture, which was able to achieve a training accuracy of just below 70%. The code for this is located here. I then pivoted to using the ResNet50 pretrained model to determine which one would give me the best accuracy. 

I was able to achieve a training accuracy of just under 80% for clothing type and color classification, which is what our use-case requirements outlined, so we will be proceeding with the ResNet model. The training is almost complete—certain clothing categories, like blazers and sweatpants, have <50 images while other categories, like tops, have >500. I need to even this out to improve the accuracy of our model by ensuring the training data is balanced. This requires me to manually label images of blazers/sweatpants with the color since I was unable to find any datasets online that have the clothing type images we need labelled with the color. My goal is to get >100 images of each clothing type into the dataset which requires more images of blazers and sweatpants. I think this will get our testing accuracy to 80%.

When training, I realized that the model was able to achieve high accuracy for clothing type and color classification but really struggled to classify the usage of clothing (formal, casual, sports). I realized that this is due to the difference in classifying clothing type/color and something more subjective like usage. Machine learning models can recognize patterns in pixel values, which help classify color, or patterns in shape, which help classify clothing type, but usage is much more subjective and has less to do with the quantitative value of an image (the array of pixel values that the model will process). We have decided to pivot to attempting to classify the usage of clothing but having the user validate our classification(and provide the correct usage if ours is correct (subject to change after discussing during capstone lab). 

I also realized that creating predefined outfits isn’t necessary because our SQL queries handle the outfit generation and don’t need to be based on predefined clothing type combinations. Since this removes one of my tasks, our team decided that I could work on the frontend instead since Gabriella has mostly been focusing on the backend and our classification model is done. 

I also worked on peer reviews and our design report this week. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the proiect schedule?

My progress is on schedule.

What deliverables do you hope to complete in the next week?

I hope to finish the design report. I am working on increasing the size of our dataset and believe I can finish it and the training by early next week.

Team Status Report for 10/5

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk that could jeopardize the success of the project is our ability to determine the usage of a piece of clothing(formal, casual, business casual). Our own model & the pretrained ResNet model both struggle to classify the usage and we believe this is due to the usage being less correlated with the image data (array of pixel values) and more determined by societal standards. This is discussed more in Alanis’ section. We have defined two possible ways of managing this risk—we thought about reverse google image searching the image and webscraping the formality, or, classifying the usage ourselves (with a low accuracy) and then having the user confirm. We are leaning towards having the user confirm but will concretely pick one of the solutions after discussing with an advisor on Monday.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

We will have changes to our block diagram and system specification once we pick one of the two alternative approaches defined in the section above. Since we will probably end up having the user confirm the usage type, our use-case requirements will likely lower for the usage classification accuracy and we will add some type of UI to allow the user to confirm/edit the usage type to our frontend, but everything else will stay the same.

Provide an updated schedule if changes have occurred.

Our schedule has not changed. 

This is also the place to put some photos of your progress or to brag about a component you got working.

Photos of our progress are located in our individual status reports.

Alanis’ Status Report for 9/28

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours.

I spent some time on my original dataset processing code, but I had to abandon it since we had to switch datasets and the new dataset that we picked needed to be processed in a different way. We switched dataset because I discovered that our original dataset did not label each picture with the attributes we needed(style, color, fabric). Many pictures were just labeled with non-exhaustive list of a few attributes. For example, most pictures weren’t labeled with a color at all, which made this dataset unsuitable for our application because color is a crucial thing we need to classify by. I then wrote code to process the dataset and train a classification model, with a common neural network architecture used for image classification. The training data and the model architecture will be fine-tuned later, this is just to have the basic structure working. With this baseline code, I was able to achieve an accuracy of 40%, which is much lower than we need.

I did research on the best neural network architecture to use for our image classification situation, and found that a convolutional one works well because it scans chunks of the image, allowing them to learn features like colors and shapes. They are translation-invariant, meaning they will be able to classify a piece of clothing regardless of where it is located in the frame. They also don’t need as many parameters as a fully connected network, which works well for our application as this will reduce the model size which is beneficial for ML on an edge device. We will use pooling layers as they make the network more invariant to small distortions, which is helpful since we are unsure of how good our camera feed will be, and because they speed up computation which will help us with our speed constraint. I chose to use the sigmoid function to determine label probabilities since sigmoid is the most suitable for multilabel classification due to producing independent probabilities for each label. I also did some research on pretrained models that we could possible use(just for risk mitigation). This information is located in the team report. I also worked on the script and slides for the design presentation as well as our design report. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the proiect schedule?

My progress is on schedule.

What deliverables do you hope to complete in the next week?

I hope to finish the model classification by achieving the accuracy desired. I hope to also start on creating predefined outfits. I also hope to finish the Design Report.

Team Status Report for 9/28

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

One of the most significant risks that we have identified is the model classification accuracy. Our initial model accuracy that was achieved from a basic image classification neural network structure and no training data manipulation, was about 40%. We are going to augment our dataset by applying rotations, translations, and flipping. We are also going to fine tune our neural network structure to be applicable to our application. If this does not give us a sufficient accuracy, we will use a pretrained model like Resnet50, which we believe is the best option due to its high accuracy and smaller size than other pretrained models like Resnet101. It is also available with Tensorflow, which we are using.

Another risk we identified is storing the images in our database as originally planned. Based on research, it seems like this would not work well because it can lead to a very increased database size and performance issues. This risk is being managed by saving image paths in the database instead, and having the actual images saved on AWS S3. S3 seems to be ideal for our use case because of its scalability, availability and performance, but if this does not work out, we will look into other file storage systems.

 As for the hardware side of things, there are no significant risks as the modules and desired functionalities are possible on the Xavier NX. One moderate risk is that the processing speed and RAM capabilities would be too low to properly use the model and potentially host the mobile app. We believe that with the proper optimizations, it should be possible, but one thing that we are doing to mitigate this potential issue is by hosting the mobile app on a server if the speed requirements cannot be met on the Xavier.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

Because of the risk with storing images in our database, we decided to use the managing plan and are now going to be using AWS S3 to store our images. It will cost $0.023 per GB per month. We aren’t sure how many GB we will need yet, but the overall price should be minimal. Additionally, this change requires moving a few things around in our Gantt chart to have time to integrate AWS S3 to our application.

We changed our image classification accuracy requirements based on our dataset changes. We are now classifying images by clothing type, color, and usage(casual, formal) since these are labels present in the dataset. We want an 80% accuracy for clothing type and color since this will allow us to generate a majority of accurate outfits while giving us a margin of error for sweater vs sweatshirt. We want 70% accuracy for usage type since this is just meant to be something to help us determine the formality of a piece of clothing, we are still focusing on creating predefined outfits and basing formality off of those.

No hardware changes have been made, some aspects were solidified further, such as the Xavier NX and development environment to be used (Ubuntu 22.04)

Provide an updated schedule if changes have occurred.

Alanis has switched the order of two tasks: model training and predefined outfits. The model classification is a much more difficult task and has the possibility of taking longer than the initial week we had outlined, while creating predefined outfits is not a difficult task and time-consuming but not a main priority right now. Our updated Gantt chart is located here.

This is also the place to put some photos of your progress or to brag about a component you got working.

Photos of our progress are located in our individual status reports.

Part A: Written by Alanis

Our product solution is a solution for a problem which doesn’t really fall into the category of public health, safety, or welfare. Style Sprout, solves a “first world problem”, since it provides outfit recommendations with people’s closets, and finding a good outfit in the morning doesn’t significantly impact your physiological/psychological wellbeing, protect you from physical harm, or provision basic needs for you. There are two ways in which Style Sprout minimally relates to public health. Style Sprout shows users various stylish outfits that already exist in their closet. This is done with the intentions of dissuading people form purchasing cheap fast fashion by reminding them about the great clothes they already have. This benefits public health as fast fashion contributes to global warming and pollution, which harms public health. Style Sprout also notifies first-time users about the way in which we use their data, where we will take pictures of their clothing from a camera feed and input them into a machine learning model, which classifies them. One image of their clothing will be stored in our dataset. This notification ensures that users are aware exactly how we use their data in conjunction with artificial intelligence, which could benefit the psychological health of people who are sensitive to the way their personal data is used. We have ensured that our solution does not harm their safety since the Jetson Nano is a small and widely used computer which would not cause them physical harm, just like the camera and light attached to it. We have ensured out solution will not harm people’s basic needs since it does not relate to food, water, shelter, etc. at all.

Part B: Written by Riley

There is a distinct social need for aid in dressing and fashion. Many people wish to dress better or more appropriately and do not have the capabilities or time. Our product solution can address this need by providing an easier way to achieve this with clothing specific to their closet. Most products currently do not provide an easy way to categorize clothing and require uploading of images, something that could form a technological barrier for some. Our design would allow the users to scan in their clothing quicker than alternatives, making it pertinent for users with minimal time due to caretaking. 

Another social issue that our product solution could help alleviate is socioeconomic. By focusing on the user’s own closet, it allows them to make more out of their closet than what would otherwise have been possible. This is critical for many families by allowing them to limit their clothing bills which is a large portion of many budgets. By generating outfits whether they be typical or novel, it allows the user to gain inspiration in styling and use of previously underused clothing. By allowing the user to make better use of all their clothing, it limits the desire to expand their wardrobe by providing novel options out of existing pieces. Additionally, by better utilizing their clothing, it spreads wear and tear out among many pieces of clothing, limiting damage, easing repairs and generally extending the lifespan of clothing.

Part C: Written by Gabriella

Our product solution, Style Sprout is designed to help users make full use of their existing closets. This reduces their unnecessary clothing purchases and also encourages fashion sustainability. By simplifying the process of choosing an outfit, our solution helps to combat fast fashion which has terrible economic effects. Additionally, as our application helps to prevent consumers from making unnecessary purchases, not only do our users save money but also the more people use it, the lower the demand for fast fashion becomes.

Economically, Style Sprout helps to make both individual and global impact. For individual users, people benefit by reducing their need to purchase new clothing and it also helps them really get full usage out of pieces they already own. More largely, our app helps to decrease the need for fast fashion overall which can lead to less wasteful consumption of clothing. Finally, our product itself does not create a large economic strain as it uses a camera, cloud storage, and a server. Therefore, Style Sprout helps users be more economically conscious and promotes economic sustainability to individual users, as well as it encourages more mindful fashion consumption overall.

Alanis’ Status Report for 9/21

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours.

I was in charge of our group’s project proposal presentation so I made many intermediate and final revisions to our slide and practiced the presentation many times over the weekend. Two of those presentation run-throughs were in front of my team members to get their feedback. I then presented our proposal on Monday 10/16 to about ½ of the Fall 2024 capstone class. I also worked on our abstract and gave peer reviews to other capstone groups.

I began my implementation of our classification model. I created a bare-bones implementation of a clothing classification system using the fashion_mnist dataset provided by Tensorflow, just to create the general structure of my code. This file is linked here. I am also working on processing the DeepFashion dataset into training/testing data to replace the placeholder fashion_mnist data. I needed to process each dataset image by changing its name to include the clothing type, like dress_001.jpg. Previously, each image was located in a folder named with the clothing type and was named with a number starting from 1, which led to many clothing items having the same name(001.jpg, just located in different folders). I also created a labels file to add tags to each image since this is a multi label dataset. I will be using the pandas library to help process the training data and I finished matching each image name to the clothing type.  I wrote a script to do these things, which is linked here. The files I created are located in our team repo. I was struggling to download the color/fabric file because it appears as a many -1s instead of the actual attributes. I will troubleshoot this and find an alternative way to download the file.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the proiect schedule?

The progress is on schedule. I am trying to finish the dataset processing before the 25th, when I am supposed to start creating predefined outfits, so that I can use the training/testing data to create them.

What deliverables do you hope to complete in the next week?

In the next week, I would like to have finished processing the dataset so that I can begin fine-tuning the model for accuracy. I would also like to finish creating predefined outfits for each category(warm business casual, cold casual, etc.)

Team Status Report for 9/21

  • What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

One of the most significant risks that we have identified is the integration of all the separate parts of our project. We think that integrating our classification to be stored in a database which our backend can pull from will be difficult considering all the moving parts. We are managing this risk by setting early deadlines for our individual pieces so that we have a lot of time to integrate. The contingency plan for us not being able to integrate in time would be to simplify our project to make integration easier. 

Another significant risk has to do with the clothing classification for our project. Currently, the dataset we are using is displaying -1s for all of the attributes. We are managing this risk by troubleshooting with the file, however if this does not work as a contingency plan we will look into using alternative datasets that still classify clothing based on item, fabric, and color.

  • Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

We are going to lower the requirements that we created for classification accuracy, because they were set very high and according to faculty feedback, are too high to be feasible. Instead of the 98% clothing type accuracy, 100% color accuracy, and 85% fabric accuracy, we’ll be looking for 90%, 95%, and 80% respectively. Since our classification requirements have been changed to lower the accuracy we are looking for, we will also lower our requirement for outfit correctness to 80% from 90%. This is because item classification accuracy plays a large part in outfit correctness, so we must lower both accuracy/correctness thresholds. Additionally, we have started to finalize our hardware decisions and have narrowed it down to a few suitable choices for each part.

  • Provide an updated schedule if changes have occurred.

The github setup task was reassigned from Alanis to Riley. Aside from this, the web app mockup task was changed to making the demo Flutter app, and the mockup task is now moved to being done with the UI animation task. We are currently on track with our scheduled tasks and assignments. 

Here is the updated schedule.

  • This is also the place to put some photos of your progress or to brag about a component you got working.

 

Photos of our progress are located in our individual status reports.