Category: Henry’s Status Reports

This week:

• Created visuals for our poster presentation. Took some inputs/outputs of our visualizer system and made images that demonstrate how the visualizer works.
• Started working on the final report. Wrote the validation section.

  For Next Week:

• Prepare for the final demo.
• Complete the final report.

I am on schedule.

This week I worked on:

• Validation section of the final presentation. Wrote about validation on each component and system validation.
• Software integrated. Integrated visualizer API with user interface (aka main). Some new issues were found during integration, which we are fixing right now.

For next week:

• Work on the final video and final report, and post presentation.

I am on schedule.

This week I worked on:

• Software Integration of Visualizer
  • Integrated all components of the visualizer API. It was running very slowly as there’s a fixed cost every time we use the ML model for classification. To fix this, I created batch classification methods. They increased the runtime per image from 4000 ms to 10 ms.
  • Webscraper integration required an algorithm to determine how well two given labels (one for user’s clothes and one for online clothes) matched. I created my own algorithm that took a weighted average with more weight given to more confident labels. It also uses L2 error for color.
• Validation Testing for Visualizer
  • Validation testing on the image and labels storage made me realize 1. storage hits were extremely unlikely because the keys was based on specific labels and color. and 2. the storage was too large to fit in memory.
    • 1. color keys is now rounded to closest color our of 125. Only most confident label is used for storage.
    • 2. Instead of an in-memory storage, it will be on-disk with memory-caching using DiskCache.
  • Core functionality of Visualizer is quite good, as seen in interim demo.
• Final Presentation
  • started working on the final presentation.

For next week:

• Complete software integration with matching api, user preference model, and user interface.
• Complete software validation.
• Keep working on Final Presentation

I am on schedule.

This week I worked on:

• created new clothing classification model to reduce overfitting. Added stronger preprocessing, dropout layer, batch normalization layer, and trained for less epochs. The new model achieved ~70% validation accuracy.
• created top-5 accuracy test on test data set. Overall 91% top-5 accuracy (yay! this meets our requirements of 90%)
• Started software integration of visualizer API. Integrated clothing detection and classification, working on integrating webscrapper right now.
• Helped Fred with hardware design. We required high precision in assembly so I suggested using laser cut components to aid in assembly. For next week:
• Finish visualizer software integration and create validation tests for software component.
  • measure accuracy on images scraped from online
  • measure runtime of system
  • measure if storage costs are within our initial predictions

I am on schedule.

This week I worked on:

• fine-tuned detector. It didn’t result in significant increases in accuracy.
• I ran into some overfitting problems with the classifier. The training accuracy would hover around 70% but the testing accuracy was only 30%. I added a batch regularization layer, dropout layer, and better preprocessing. Hopefully by next week I can get a model with good accuracy. If I still can’t get a good model trained, as risk mitigation I can find an existing model online, however it is not ideal as existing models use old architecture so I would not maximize accuracy.
• implemented clothing images and attributes storage. Uses pickle to store images and attributes in a dictionary.
• Implemented color attribution. Uses MMCQ algorithm with color thief API.

For next week:

• Retrain classifier with new model.
• train fine-grained clothing attribution model.

Although I am slightly behind schedule for the models, I am ahead of schedule for the clothing storage, which means I can focus more time onto the models.

The past two weeks I worked on:

• Finished the design report. I wrote most of the documentation on the visualizer component.
• formatted datasets for training. Detector required tfrecord files from DeepFashion2 dataset while classifier required images sorted into classes from DeepFashion dataset.
• Trained a clothing detector that detects bounding boxes and does simple classification of clothing articles.
  • tried yolov4 using pytorch, but API was really bad and had many problems I had to fix by looking at pull requests. Ultimately, didn’t use.
  • tried efficientDet0 with tensorflow2 object detection API and it was much better documented and had more community support. Ultimately, used this.
  • Using DeepFashion2 dataset, which doesn’t provide good classification labels, but has bounding boxes over every clothing item (DeepFashion only has 1 bounding box per image)
  • unfortunately, I didn’t measure accuracy during training, but here are the loss figures:
  • 
• Training a clothing classifier that provides more fine-grained classification.
  • Using efficientNetb0 from keras. Below is test accuracy:
  • 
  • Still requires more epochs until convergence.
• Created simple clothing recognition API with aforementioned models. Currently returns top 10 most likely bounding boxes and labels.

For Next Week:

• Finish training models and perform fine-tuning.
• Start training models for more fine-grained classification like shape and material.
• Implement color classification.
• Right now detector could predict differently from classifier. I’m not sure what to do when this happens, but I should figure it out.

I am slightly ahead of schedule.

This week I worked on:

• Presented the design review presentation. We got a lot of good feedback, especially for our hardware design and our ML algorithm.
• Worked with Fred on details of hardware design. Came up with a design that uses gears so the servo doesn’t need to be in the middle of the axis. This ensures there’s no weight on the servo and it also allows us to potentially add additional servos if we need more torque.
• Wrote code that can train any prebuilt models on pytorch. Uses fastai wrappers to make the job easier.
•  
• Created a resnet34 classification model with 66% top-1 test accuracy. Not a metric worth noting as I still need to have better data augmentation and hyperparameter tuning. We also need validation testing from internet sources and we’re looking at top-3 accuracy, but it’s a start.

For next week:

• Finish the visualizer section of the design report.
• Bootstrap training data with additional data from the web.
• Create validation set by manually labeling web-scraped images.
• If I have the time, try training an object detection model that can find top and bottom.

I am on schedule.

This week I worked on:

• Created the high-level software design for most components apart from the robot’s software. Here’s a block diagram:

• Started specing out implementation details for several software components: https://docs.google.com/document/d/1tBEqCWRspqwA_sYQ4ymcQYzh6Qb36l5V6L4Wjh0QcRo/edit?usp=sharing
• Worked on the design review slides:
  https://docs.google.com/presentation/d/1kY1kIH_YxvDuDWgsggm8xlKYttMxBnfFL2lo6iBvHSg/edit?usp=sharing

  For Next Week:

• Practice for design review presentation. Incorporate the feedback into our design review report.
• Work on the design review report. The first link above already contains a lot of information on the software design. Fleshing it out with more details and formatting it should suffice for the report.
• Lay out the software system with stub API calls. Decide on which deep learning architectures to first try training our recognition model on.

This week I worked on:

• Rescoping our project because our original proposal was too difficult. Our initial hardware design had too many moving components and required extensive CAD design and machining in order to create and assemble the parts. After discussing with professor Mukherjee, we created a new, more detailed hardware design linked here:  https://docs.google.com/document/d/1n_NDpZC0ni2qttofl1uJjUk9jUlInqgykfnHyiBxZws/edit
  This design requires little to no CAD or creating new parts, using items we can purchase online. I wrote the documentation based on our discussions as a team and I made the torque calculations.
• looking into what motors we need for our design. I included this into the documentation, but we are looking for a 360 degree servo with at least 1 Nm of torque.

  For next week:

• Decide on the exact parts we need for our design. We want feedback from course staff of our design first before we do this. In our current design, we are looking for:
  • Turntable bearing
  • Metal-metal epoxy
  • Metal Ring
  • Servo
  • Controller
  • Base
• Flesh out our software design. I will look into what libraries are available online for web scraping and clothing recognition and decide which ones we will try to use. I will also create the API between our software components, defining what each component needs to be able to do and how our system works together.
• Prepare for the design presentation. We will work on the slides as a group, but I will be presenting it.

Henry’s Status Report for 02/20 This week:

• Worked on fleshing out our project scope and planning.
• defined problem area and use cases -worked with team on defining specific requirements
• identified technical challenges for software requirements
• created initial software design flowchart

Next Week:

• clean up presentation for Monday
• flesh out solutions approach