- Finalized the mechanical design for the conveyer belt system (currently have rack and pinion as a contingency plan).
- Extracted color ranges from hsv space for certain images (experimented on a picture of a banana).
- Ordered the Nvidia Jetson Nano 2GB and the Raspi camera module V2. Started experimenting with them. Next step is to write a program to take a picture and save it on disk.
- Worked on the design presentation. Finalized some design points (apart from mechanical) that were previously unanswered.
- Performed risk analysis and risk mitigation. In particular, if we can’t assemble the conveyer belt ourselves, we will use the treadmills in the CUC gym to simulate the process.
- Started looking into algorithms for extracting features (e.g localized black spots in a banana) from the segmented images.
Ishita Kumar’s Status Report for 03/06/21
This week I talked with the team and decided on parts to order. We sent in an order for the Raspberry Pi camera and the Nano and received it already. I worked on image segmentation on a picture of a banana from google. I converted the image from rgb to hsv space and extracted colors within a certain range of yellow using a mask. I am going to continue working on improving the mask by using it on different pictures of bananas. I also worked on the design presentation slides. For next week, I am going to improve the segmentation algorithm and add to it to segment black spots on bananas as well. Then, I will try to merge my code with Ishita Sinha’s so we can do color analysis.
Kushagra’s Status Report for 02/27/21
- Watched tutorials on the jetson nano and learned about all the machine learning / AI capabilities the board has to offer. Found the sdk to be very powerful.
- Decided to move away from the rotating plate so that we can incorporate non-round fruits (e.g. bananas)
- Developed frameworks for pixel isolation (segmentation and gaussian distribution).
Team Status Report for 02/27/21
We worked on the project proposal and finalizing our mechanical design this week. We decided to not use the rotating plate for placing the fruit anymore as it would not support non-round fruits, like bananas, very well. We, instead, are planning to use a diagonally placed 2-camera system to take pictures of the fruit. We have also decided to use pistons to push the fruit in the right direction to physically sort it.
We researched pistons, specifically solenoid-type pistons, on Prof. Savvides’s suggestion, but we found them to be quite expensive. We are a little worried about using pistons as none of us have much of a mechanical background so we will discuss our mechanical design again tomorrow.
As of now, we are slightly behind schedule as we had wanted to order the parts this week but we think it is better to talk and share our parts list and reasonings to the professor and our TA on Monday before buying things. So, this is not too concerning as it is better to double-check before spending too much money. Ishita Sinha and Ishita Kumar have started looking at color analysis and image segmentation and should code up a simple working sample by next week. Kush will meanwhile take pictures for training data. If we get all this done next week, we will be very well on track.
Ishita Kumar’s Status Report for 02/27/21
This week I helped my team prepare for the project presentation. I researched papers on automatic rotten/fresh fruit sorting to propose better ideas for our mechanical design. We decided to get rid of the rotating plate idea for now due to it not working well with non-round fruits such as a banana. We plan to go for a diagonally placed 2-camera system.
I also researched pistons and in particular, solenoid-type pistons due to Prof. Savvides’s suggestion. They were more expensive than expected, ranging from $245 to $436. I also have started looking up code samples for image segmentation and plan to implement a simple one in the first half of next week. I think we may be slightly behind schedule but not too shabby. The mechanical design, particularly the pistons worry me. So, I plan to meet with the team to go over our design again.
Ishita Sinha’s Status Report for 02/27/21
This week, my teammates and I have been working on finalizing our final design and determining our parts list accordingly. We plan on getting this approved on Monday. Attached below is the preliminary design I drew out for our product:
Besides this, I’ve also been looking into the color analysis of bananas to write up the code for a rottenness predictor based on color. So far, I’ve primarily researched methods for this and have tried to understand how color analysis is performed. I’ve also looked into how we can develop a rottenness predictor given a dataset of images classified as rotten, ripe, and unripe.
We are behind what we had planned in our schedule in that we haven’t yet ordered parts, but I believe that’s okay since we are waiting to confirm our parts list on Monday, after which we’ll order the parts. Besides that, in terms of working on color analysis and starting work on rottenness prediction, I believe I’m on track.
In the upcoming week, I hope my team and I can finalize the parts list and order it. I also plan on downloading Google images of bananas, making the background white if it isn’t already, and developing a dataset of rotten, ripe, and unripe bananas. I also want to work on the color analysis algorithms to train the dataset so that my team and I can then test the algorithm’s predictions before we develop our actual dataset.
Kushagra’s Status Report for 02/20/21
- Decided on bananas as the primary fruit since discoloration is more prominent and easier to detect. Still need to learn about HSV.
- Started brainstorming ways to isolate fruit through segmentation and detect localized pixels of different colors. This method seems easier than NNs, so preferring this, but keeping options open.
- Also started brainstorming on how to differentiate between fruits based on pixel distribution.
- Preparing the slides and talking points for proposal presentation
- Started looking into Jeston Nano Tutorials and JetPack sdk.
Team Status Report for 02/20/21
For now, it seems like the most significant risk we’re undertaking is visualizing the design to the best of our abilities, but not actually being able to see it physically before us. We believe our design is fairly complicated, so there are many aspects of it that could go wrong or force us to rethink our entire design if it doesn’t work out as we expect it to. As a contingency plan, for now, we plan on rotating the stand holding the fruit manually, and/or clicking pictures manually, in order to at least be able to make progress while rethinking our design.
We modified the product to involve holding the fruit up and rotating it using a rotating disk of sorts. We did this because we plan on using a banana for our MVP, so a banana is a fruit we’d like to hold up in order to capture pictures from all sides.
We have a preliminary schedule planned for now, which we will be putting across in our proposal presentation.
Ishita Sinha’s Status Report for 02/20/21
This week, in our weekly team meeting with Professor Savvides and our TA, Uzair, we worked on reasoning about the feasibility of our project and what we could modify about our proposed MVP to make it better suit our needs. We realized that since bananas rot quite quickly, using a banana in the MVP would be a good idea. We also discussed ways in which we could get pictures of the fruits from all angles. Another idea that was brought up was that we could predict the rottenness of the fruit, or how long it was expected to last, and this sounded like quite an interesting idea for our final project!
Initially, we were considering only round fruits in order to be able to cover all of the sides with ease using a rotating disk. However, when we started considering bananas, we shifted our focus to find ways in which we can hold the fruit up and click pictures from all angles to check for rotting. I spent some time this week thinking about the design of our model. We could have a holder that could hold our fruits up, and then we could have a camera on a rotating stand that could click pictures of the fruit from all angles.
However, this would imply we need quite a heavy-weight stand to hoist the camera on since rotating the stand would not be easy. Thus, an alternative is that we could have the holder rotate the fruit. For this, I looked into rotating disks. To the rotating disk, we could attach a hook to hold the banana (or any other fruit), and then, as the fruit would rotate, the camera would capture pictures. When I looked up rotating disks online, I usually found ones that looked like the one below:
Such a rotating disk would’ve suited our earlier model with round fruits, but it doesn’t work for our current model. We would want rotating disks that could have a hook attached to them using which we could hang the fruit. Thus, I thought we would probably need to 3D print the rotating disk so that we can attach a hook to it, or design it such that it already has a hook built into it, from which we can hang the fruit.
We seem to be on schedule for now. Over the next week, first off, I plan on updating our project proposal presentation in order to ensure we are putting across our idea clearly. Post that, I plan on looking into image segmentation and edge/color detection algorithms so that I understand them better since we would need these algorithms not only to identify the fruit but also to be able to predict rottenness.
Ishita Kumar’s Status Report for 02/20/21
This week, in our initial meeting with Prof. Savvides and our TA Uzair, we were recommended to implement FarmFresh MVP with bananas as they rot quite quickly and noticeably. Since our FarmFresh design as of now plans to use a rotating plate for fruits to be placed on, I researched ways to hold up a banana. I found a banana holder that could fit our needs in the market, but we may have to design a transparent or thinner version of this. Alternatively, there are hammocks or hangers for bananas that can also hold a single banana. For now, we plan to just manually take top-view pictures but different angles will be important later on. We are on schedule as of now. For next week, I plan to research ML models, that will work well with recognizing fruits. Specifically, I will look into Image Segmentation and try to implement a simple program that can detect bananas in Google images of them.
