Posted on

Team Status Report – 12 Feb 2022

This week, our focus was on finalizing our project idea, ironing out various details and working on the proposal presentation. Early on, we had already finalized our idea of a smart-fridge add-on that would use computer vision (CV) to keep track of fresh produce going in and out of the fridge, and remind users + suggest recipes accordingly. The key technical challenges of our project would be the CV system, database system and overall workflow between the two and the intermediate UI.

Through a combination of thinking our project through and Prof Marios’ advice, we note some of the following details that we needed to iron out:

  • Choosing the right CV algorithm: Finding several appropriate CNN networks and/or classical methods to use, and testing each one for accuracy to benchmark speed, accuracy and other metrics
  • Image Segmentation/White background for the CV algorithm: Some of the classifier networks that we initially found need a white background to work well. To overcome this, we are currently thinking of a little platform that involves a little white ‘screen’, which the user will place the fruit on.
  • UI: We are thinking of putting together a vanilla bootstrap website since the UI does not need to be particularly advanced. This will let the site run quickly.
  • Database System: We need to find a database system that will integrate well and easily with our back-end, and is sufficiently scalable for the project.
  • Back-end: We are thinking of building our back-end upon Node.js and Express, as the JavaScript framework and widespread availability of libraries make it much easier to avoid rewriting common boilerplate code and enable us to focus on writing code specific to our project. However, more research and investigation should be done before we embark on back-end development. We will also iron out our API design, which will help guide our choice of back-end architecture.

With this, we completed our proposal presentation on Wednesday! Overall, our hard work paid off, and the proposal presentation went very well. It seemed that we were on track, and our next stage was to begin our research and prototyping process, which we are currently halfway through on.

This document summarizes what we are currently planning for the architecture and how each component fits together (including documentation of API endpoints): Architecture

 

Posted on

Samuel’s Status Report – 12 Feb 22

This week, we finalized the idea for our project, and successfully ironed out some issues.

Notably, I am in charge of the CV system; we were able to find a dataset of many fruits and vegetables (Fruits360 Dataset) which we could possibly use to train our CNN classifier. It is a fairly extensive dataset, with 90483 images and 131 classes of fruits and vegetables. Following a TA’s suggestion, we were also be able to find a ResNet based CNN classifier which we could potentially use for our project, and which I am currently trying to implement.

However, Prof Mario’s observed that the dataset images consisted of a white background, which implied that if trained using these images, our classifier might not be able to detect fruits in an arbitrary background. With this in mind, I came up with the idea of using a platform and a white screen that fruit can be put on, thus allowing us to not only easily detect and segment out the fruit from the (white) background, but also allow us to use the extensive dataset.

I am also fairly proud of my contribution to the “Use Case Requirements” part of the proposal presentation, where we considered our product’s speed accuracy and cost metrics from the perspective of tangible monetary cost.