What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours.
This week, I finished the systematic testing of our classification models by testing how brightness, x/y translations, and rotations affect the accuracy of our classification models. I wrote a script to do this and also created some graphs.
I also evaluated which architecture out of ResNet50/101/152 would be best for the color and clothing type models by weighing the correct predictions vs close predictions. A close prediction for clothing type would be guessing jeans instead of trousers or a dress instead of a jumpsuit. A close prediction for color would be guessing beige instead of white or black instead of grey. The general description of the criteria I used to determine a close guess for clothing type would be the same category(tops, overwear, bottoms) and a similar shape/usage/weather (jeans and trousers can both be casual or business casual, dress and jumpsuit are both suitable for warm and neutral but not cold weather). The general description of the criteria I used to determine a close guess for color would be if the actual color could be predicted by lightening/darkening the predicted color(lighting would do this, and you could reach white by lightening beige which is already quite light, you could reach black by darkening grey, but you could not reach blue by lightening/darkening red). The exact criteria for close guesses will be outlined in our final report, however it is quite long so I will not include it here.
Also, this wasn’t done for the usage model since only the ResNet50 architecture was able to converge during training.
I also debugged our .tflite color model. We have 3 classification models (clothing type, color, and usage) that we converted to .tflite for inference on the Jetson. They are all ResNet50 architecture and trained in the same way (mostly the same data but with different labels). They all perform at an accuracy of 60-70% before conversion to .tflite. After running inference on the converted .tflite models on the Jetson, the clothing type and usage models perform with the same accuracy however the color model accuracy took a very large hit and mostly predicts blue, brown, and black.
We verified that the camera is not causing the issues since we are testing the models using pretaken images. I tested the problematic .tflite model on my computer and the accuracy is the same as the original. I verified this using 2 different Tensorflow versions, the one I used to train and the one running on the Jetson. I also wrote some code for Riley to try using the original model optimized for the Jetson using TensorRT, however this was very slow. I also sent Riley .tflite versions of the ResNet101/152 which also performed with low accuracy. We also tried training a .tflite model on the same Tensorflow version as the Jetson which still did not address the issue.
We decided to also pursue an alternative way of determining the color by cropping the clothing image around the largest object and determining the most prominent color in the cropped image. I wrote code to do this which is located here. We will continue to try and fix the .tflite file and will see which option works better before the demo.
I also worked on our poster.
Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the proiect schedule?
My progress is on schedule.
What deliverables do you hope to complete in the next week?
I hope to fix the color model and fine tune the color prediction from pixels code by adding the determination of the main color from the most prominent rgb.
