Category: Ishita Kumar’s Status Reports

This week, I prepared for the final presentation. The focus for me this week was fixing some visibility bugs in our hsv masks and finishing up hsv image segmentation for carrots.  I also helped my team integrate the gate into our final product and we now have a fully working system. We focused a lot on testing both software and our physical components in the past few days. I have also started on the final video by collecting B-roll footage and preparing my script. I am happy that we got to finish our product. We should be on track for the Monday video and poster submission deadline. Thank you Professor Savvides as well as our TA Uzair for all your help.

Since this week was very important for the team in terms of testing and integration, I made a plan for work to be done daily for the team. I helped finish the final conveyor belt set-up and stitched the belt together. Next, Ishita Sinha and I tested the frame detector on the banana running on the conveyor belt. We had a few bugs related to how many pixels we said were related to a banana in the frame to work out but we increased the pixels according to test results and the frame analyzer works now. Next, the hsv mask rotten detector does work for our bananas on the moving conveyor belt which Ishita Sinha and I tested too. But it heavily depends on lighting conditions. We found during testing that sometimes there is too much noise being picked up by the hsv filter masks when there is improper lighting and so we need to obtain better lighting. Next, I helped design the first prototype for our gate in CAD and 3D printed it. Unfortunately, our original design with the gate blocking the fruit did not work because the gate was too short. We have come up with another similar design with a much longer gate and are hopeful it works. I designed the new gate in CAD as well. As of now, it is my responsibility along with Ishita Sinha to fix the lighting issue so we can do a full software+conveyor belt run without issues. Overall, our design is working though. According to my plan, we should have had the gate ready by today but we had some delays because of our old design not working. This is okay for now, as we have the other design ready and we have the slack time to get it to work.

 

This week, I implemented image segmentation for both fresh and rotten apples and oranges using appropriate HSV ranges. Segmenting apples was quite difficult because of their diverse nature and the rotten parts being various shades of brown which are close to golden hues found in some apples. So, I have decided to focus our scope on a particular type of apple, probably red apples. This will help us narrow our image segmentation algorithm to work and improve our accuracy. Oranges did not pose the same issue so the algorithm worked well for this fruit. I am on track with my code for now and will fix up a few more things while Ishita Sinha tests our combined integrated code on a custom dataset. We will be meeting every day for integration and testing from now on.

This week, I focused on testing our algorithm on real bananas in real-world conditions for our project. I used white lighting as we plan to do for our final set-up to take pictures. The results were promising. I tested on healthy bananas, and our algorithm correctly identified those bananas as not rotten. I have been thinking about the parts we need for our final set-up and trying to design what we need so we can order the final set of parts for our shed set-up. I have also started looking into other fruits that we are going to use so we can have our algorithm work on those fruits as well and looking into how to use object detection so our algorithm can autonomously detect which fruit it needs to sort. I am also preparing with my team for the interim presentation.

This week, I used CAD to design the parts necessary for the rollers to be held in place to the wooden slabs in our conveyor belt and our motor. We then 3D printed our parts and they came out to fit perfectly for our needs. I also worked with Ishita Sinha to test our pixel analyzer algorithm on various images of fresh and rotten bananas found from an online dataset on Kaggle. With some trial and error and fine-tuning of the percentage of bad parts for classifying a banana as bad, our algorithm detected almost all bananas correctly for both good and rotten bananas. We decided 20% brown-black parts in a banana image was a good benchmark that worked for us. The few images that were not classified correctly had inappropriate background colors for our need such as yellow or brown. We plan to have a white background and good lighting so we are not worried about those failures as they do not apply in our case. We also acquired free rotten bananas from a store and now we are going to test our algorithm on those bananas. This next week, I want to finish our conveyor belt and decide whether to use AlexNet or not as we may not need it for our purposes. We are slightly behind track on the conveyor belt as I hoped to get it working this week but we are close and I plan to ramp up and ensure we have it working soon next week.

This week I met up with Ishita Sinha to work together on our rottenness detector. I improved the mask for detecting rotten bananas through the help of plotting rotten bananas on an HSV graph and then testing values on images of rotten bananas. My algorithm now has two separate masks one for rotten and for good parts of the banana and I worked to separate the two masks as before I just had a single mask for the whole banana. Ishita Sinha and I discussed ways to improve and implement our rottenness classifier and we decided using the areas found in our image segmentation masks would be a good start to calculating the percentages of rotten parts in fruit so we are implementing that. I also researched AlexNet and double-checked an online Kaggle dataset I found earlier to see if it would fit our needs and I believe it would as of now. I also worked with my team to build the conveyor belt this week and we made good progress with the woodwork and we will continue working on it early next week too. I am on schedule for my tasks.

I worked on improving my image segmentation algorithm this week. I noticed significant improvement from last week when using HSV color thresholding. This is because I decided to graph the HSV range in one banana image and then used information from that graph to decide my thresholds, instead of manually fixing the range through trial and error. This HSV color threshold does well on multiple fresh bananas too. I tried my algorithm on different images of bananas in different positions for testing it. I now need to work on segmenting rotten bananas as well. Ishita Sinha and I are going to merge our code and then work on classification. I am researching classifiers which I will discuss with Ishita Sinha.  I will also work on the design document due next weekend and our team is going to meet to discuss this. We want to finish the classification code for rotten vs fresh by next week.

 

 

This week I talked with the team and decided on parts to order. We sent in an order for the Raspberry Pi camera and the Nano and received it already. I worked on image segmentation on a picture of a banana from google.  I converted the image from rgb to hsv space and extracted colors within a certain range of yellow using a mask. I am going to continue working on improving the mask by using it on different pictures of bananas. I also worked on the design presentation slides. For next week, I am going to improve the segmentation algorithm and add to it to segment black spots on bananas as well. Then, I will try to merge my code with Ishita Sinha’s so we can do color analysis.

This week I helped my team prepare for the project presentation. I researched papers on automatic rotten/fresh fruit sorting to propose better ideas for our mechanical design. We decided to get rid of the rotating plate idea for now due to it not working well with non-round fruits such as a banana. We plan to go for a diagonally placed 2-camera system.

I also researched pistons and in particular, solenoid-type pistons due to Prof. Savvides’s suggestion. They were more expensive than expected, ranging from $245 to $436. I also have started looking up code samples for image segmentation and plan to implement a simple one in the first half of next week. I think we may be slightly behind schedule but not too shabby. The mechanical design, particularly the pistons worry me. So, I plan to meet with the team to go over our design again.

 

This week, in our initial meeting with Prof. Savvides and our TA Uzair, we were recommended to implement FarmFresh MVP with bananas as they rot quite quickly and noticeably. Since our FarmFresh design as of now plans to use a rotating plate for fruits to be placed on, I researched ways to hold up a banana. I found a banana holder that could fit our needs in the market, but we may have to design a transparent or thinner version of this. Alternatively, there are hammocks or hangers for bananas that can also hold a single banana. For now, we plan to just manually take top-view pictures but different angles will be important later on.  We are on schedule as of now. For next week, I plan to research ML models, that will work well with recognizing fruits. Specifically, I will look into Image Segmentation and try to implement a simple program that can detect bananas in Google images of them.