For this week, I focused on creating a prototype of the model we will use for detection and classification of grocery items. To accomplish this, I trained an initial YOLOv5 model on a small preliminary dataset of grocery items. I made use of a diverse dataset, in order to ensure basic object detection and classification functionality.
After training, I did some initial tests, using an image of a cake within the fridge that we are using. I ran the model on the image to evaluate the performance of the prototype model, below are the results.
As can be seen, the preliminary model was already quite successful in identifying the bounding box of the item, though the classification was not completely accurate, it is understandable since it was a niche item(cake from Giant Eagle).
I am currently making good progress with respect to our Gantt chart, and have started training the model slightly ahead of schedule. For following weeks, I aim to continue data collection by finding more datasets of fridge images, as well as exploring training with annotated data of images of groceries taken within our fridge. I also aim to fine-tune the model through adjusting hyperparameters and increasing training size through data augmentation to increase accuracy. I will also experiment with different variations of YOLOv5 models to see if a larger model will yield better accuracy without major latency trade-offs. I will also measure latency in terms of local and cloud run inference, in order to see which one better suits our requirements for cost and latency.