Posted on

Team Status Report 10/1

The most significant risks that could jeopardize the success of the project at the moment is we are falling behind on our initial schedule. We are a bit indecisive and because both Jiyeon and Rachana don’t have a solid background in image recognition, we could not come up with a solid design plan, and every time when we talked to the professor and the TA in the status meeting, we were not sure what would be the best solution and how to tackle the obstacles. However, we are confident that we can make much more progress as we have a solid design plan set up. One of the concerns was a hardware design. Last week after the proposal presentation and meeting, we realized that we need a conveyor belt to scan the product one by one. Raymond looked for a conveyor belt that we can use, but he couldn’t find one with a reasonable price. Professor Mukherjee showed us last year’s capstone project that built their own conveyor belt and suggested us to build one. Thus, we will be building our own conveyor belt for our project and we decided to get rid of using a weigh sensor.

This is a brief sketch of our design with a conveyor belt:

Now we have decided that we are going with a conveyor belt and no weigh sensors and we have a dataset for retail product, we are feel confident to get back on schedule– especially because we initially planned our schedule conservatively with more extra time.

Here’s our updated schedule until the design presentation. We did not need much change as we initially planned our schedule with lots of flexibility.

Posted on

Rachana Status Report (10/1/2022)

What did you personally accomplish this week on the project?

 

I contributed to the design proposal slides, and thought about the CV Design system, and how it will look like. I have been watching a few videos on people’s image classification model tutorials on Tensorflow. I have also been reviewing previous design proposal slides, and trying to come up with a feasible tested design for the CV system. We met with the professor a couple times, and this gave me more clarity on what our design should entail including more detail for the CNN including Batch norm layers, ResNet options, and what sort of pooling we should go for. 

 

Is your progress on schedule or behind ?

I would have liked to finalize the dataset a little earlier. The interesting part is we are using a Chinese dataset as for the main features, the characters dont matter at all. Progress is a little behind schedule, and I would have liked to figure out a pipeline for the OCR part as well.

 

What deliverables do you hope to complete in the next week?

I hope to get a tested pipeline, and run just 3 classes of images with that including removing shoddy images with OpenCV, and using Keras, Tensorflow. I hope to have figured out details for the OCR part as well. I also hope we have incorporated changes from our topic proposal to our design proposal while I present.

Posted on

Raymond Status Report 10/1/2022

This week I have been primarily working on drawing a new diagram for the converyor belt.  How I approach this was first watching some youtube videos and observe what the team did last semester.  Then I write out a sketch of some of the key mechanical components that I want to buy. Since I am quite inexperienced with the layout of the conveyor belt, I consulted with Tamal on some technical terms, and how to search thing up. Another important idea I learned this week is that it is very important to look at the technical specification of the component I want to buy because the size of the component has to match.

Posted on

Raymond Status Report 9/25

This week I am  working on rehearsing the design presentation and finanlizing our block diagrams. I spent a long time drawing the architecture of the system. Some of the problems I encountered this week are the architecture of the system are not that detailed. For instance, what kind of algorithms should we run on the database. How do we retrieve the data from the database etc. Also, this week I spent a long time on researching the specs of the weight sensor. However, we realized that the weighing the items may not be a feasible plan because non perishable items usually do not to be weighed. Hence, before I work on anything, I really need to think more and communicate more with my teammate. After that, I decided to learn more about the spec of the conveyor belt and reserach on how the camera fits the camera mount.

Posted on

Jiyeon’s Status Report 9/24

Earlier this week was mainly dedicated for proposal presentation. We met and discussed about the proposal slides and worked on it. After the presentation, I did more research about the image processing and OCR. I was not able to achieve all of the plans for this week because I was out of town for the conference, however, I am planning to catch up the works I missed— I will be testing some algorithms to see which might suit for our project. Also, I found just one dataset for convenience store items so far so I will look for more. After the presentation on Monday, we got a feedback about detecting a canned goods, which is hard for us to detect unless we have cameras on the side. So for next week, I will be meeting with Rachana and Raymond to discuss about the specific design plan and try to improve our plan from the feedback we got from the class presentation. We initially planned to meet this week, but we couldn’t due to our schedule conflict.

Posted on

Rachana Status Report (24th September 2022)

I finalized slides for the powerpoint presentation. This helped me very well with the brainstorming process. I came up with an FSM to help with the timing of all our objects and wait for them to be placed to scan the counter to classify images. We also wait for all objects to be removed from the counter to calculate the cost. I need to run the FSM with another TA and a professor to see if we can go with this idea.

FSM right now

I was not able to get the specifics of the neural network yet. With the limited research I have done, I think this looks feasible : https://medium.com/thecyphy/train-cnn-model-with-pytorch-21dafb918f48 .

I look to be testing this with 2 or 3 image types this week and seeing whether we can use this neural network design strategy for the other images. For the general CV design I was trying to figure out a way to generate a feature similarity index metric where above a certain threshold, it doesn’t make sense to classify the image based on how it looks, and we can just read the text off of the image. Initially I wanted to come up with the metric before, but based on some reading I realized that it might make sense to just run text detection and image classification on all the objects, and then based on the data collected, identify which conditions it might make sense to go for text detection or image classification. Based on some recommendations, I decided to go with the Multiplexed OCR for the text detection part. I think I am on schedule right now, but  I do want to get alot of the work done this upcoming week for the CV part of our system. I want a working neural network this week from pytorch, and group together multiple datasets for the convenience store images to decide on a group of 15 objects that we can test on. 

 

Posted on

Weekly Status Reports (24th September 2022)

Weekly Status Report (Team)

After talking to our TAs we need a conveyor belt to ensure that we can put objects one by one to allow the camera to scan the objects, and allow the weight sensor to work. We are planning to combat this problem by buying a conveyor belt and integrating it with our counter, and scanning technology. Raymond decided to use this conveyor belt, and he finalized the shopping budget for parts. We decided to go with testing multiple cameras from the inventory to see which gives us the best quality and definition. We also decided to go with 2 or 3 cameras to cover most of the orientations of the object. This is why we needed a conveyor belt to tell us we are not taking the picture of the same object from different angles and counting that as 2 different objects. Rachana identify more problems with our technology including orientation of objects, and the white background. We decided to go with non-perishable items as this is easier to test out as well and we don’t have to throw out objects after testing them for a week. 

 

Posted on

Introduction and Project Summary

The self-service technology innovation in food markets has led to the shopping activity being served by the shopper without relying on any service provider. Currently, self-checkout systems at top retail stores allow us to scan barcodes or look up codes for produce, and calibrate a cost. We want to create a touchless kiosk checkout system, a retail point of sale system that uses computer vision technology to identify consumer packaged goods, and non-packaged produce. Our system will contain a weighing scale kiosk with many inbuilt cameras to segment and classify produce easily, and have a wireless interface to make a more modular application.