This week, I prepared for the final presentation. The focus for me this week was fixing some visibility bugs in our hsv masks and finishing up hsv image segmentation for carrots. I also helped my team integrate the gate into our final product and we now have a fully working system. We focused a lot on testing both software and our physical components in the past few days. I have also started on the final video by collecting B-roll footage and preparing my script. I am happy that we got to finish our product. We should be on track for the Monday video and poster submission deadline. Thank you Professor Savvides as well as our TA Uzair for all your help.
Team Status Report for 05/01/21
This week was very important for our team as it was around the 2-week mark before the final deadline. So, we followed a daily work plan for this week.
Ishita Sinha and Ishita Kumar finished the final set-up for the conveyor belt, including positioning its height and stitching the belt in place. Kush integrated the code onto the Jetson Nano. Then, IK and IS did several conveyor belt + software runs. Initially, the algorithm was too slow to meet our conveyor belt speed requirements. However, IS sped up the code from 3s/fruit to 0.04s/fruit by switching to Numpy arrays. So, now we meet our speed requirements. Some frame analyzer bugs were also fixed and the software+conveyor belt system works well for the most part. We found that sometimes, the hsv filter masks the rottenness detector uses picks up a lot of unrelated noise due to bad lighting conditions. So, it is very important for us to obtain better lighting for more consistently successful runs of our system.
The final part of the MPV left is the gate. Kush and IK designed the first prototype of the gate based on our original design. Unfortunately, the gate from the original design is too short and the banana just gets stuck and starts rotating on the conveyor belt. However, we came up with several new prototypes and have CADed a longer version of our gate which we hope will work.
Having the strict plan for this week helped but we strayed from the plan by mid-week due to unexpected bugs and our original gate design not working. However, we learnt many important things about our design this week during integration and will continue to follow strict plans for the next weeks to finish up our original MVP design (including the gate) with the addition of the conveyor belt.
Ishita Kumar’s Status Report for 05/01/21
Since this week was very important for the team in terms of testing and integration, I made a plan for work to be done daily for the team. I helped finish the final conveyor belt set-up and stitched the belt together. Next, Ishita Sinha and I tested the frame detector on the banana running on the conveyor belt. We had a few bugs related to how many pixels we said were related to a banana in the frame to work out but we increased the pixels according to test results and the frame analyzer works now. Next, the hsv mask rotten detector does work for our bananas on the moving conveyor belt which Ishita Sinha and I tested too. But it heavily depends on lighting conditions. We found during testing that sometimes there is too much noise being picked up by the hsv filter masks when there is improper lighting and so we need to obtain better lighting. Next, I helped design the first prototype for our gate in CAD and 3D printed it. Unfortunately, our original design with the gate blocking the fruit did not work because the gate was too short. We have come up with another similar design with a much longer gate and are hopeful it works. I designed the new gate in CAD as well. As of now, it is my responsibility along with Ishita Sinha to fix the lighting issue so we can do a full software+conveyor belt run without issues. Overall, our design is working though. According to my plan, we should have had the gate ready by today but we had some delays because of our old design not working. This is okay for now, as we have the other design ready and we have the slack time to get it to work.
Ishita Kumar’s Status Report for 04/24/21
This week, I implemented image segmentation for both fresh and rotten apples and oranges using appropriate HSV ranges. Segmenting apples was quite difficult because of their diverse nature and the rotten parts being various shades of brown which are close to golden hues found in some apples. So, I have decided to focus our scope on a particular type of apple, probably red apples. This will help us narrow our image segmentation algorithm to work and improve our accuracy. Oranges did not pose the same issue so the algorithm worked well for this fruit. I am on track with my code for now and will fix up a few more things while Ishita Sinha tests our combined integrated code on a custom dataset. We will be meeting every day for integration and testing from now on.
Ishita Kumar’s Status Report for 04/10/21
This week, I focused on testing our algorithm on real bananas in real-world conditions for our project. I used white lighting as we plan to do for our final set-up to take pictures. The results were promising. I tested on healthy bananas, and our algorithm correctly identified those bananas as not rotten. I have been thinking about the parts we need for our final set-up and trying to design what we need so we can order the final set of parts for our shed set-up. I have also started looking into other fruits that we are going to use so we can have our algorithm work on those fruits as well and looking into how to use object detection so our algorithm can autonomously detect which fruit it needs to sort. I am also preparing with my team for the interim presentation.
Ishita Kumar’s Status Report for 04/03/21
This week, I used CAD to design the parts necessary for the rollers to be held in place to the wooden slabs in our conveyor belt and our motor. We then 3D printed our parts and they came out to fit perfectly for our needs. I also worked with Ishita Sinha to test our pixel analyzer algorithm on various images of fresh and rotten bananas found from an online dataset on Kaggle. With some trial and error and fine-tuning of the percentage of bad parts for classifying a banana as bad, our algorithm detected almost all bananas correctly for both good and rotten bananas. We decided 20% brown-black parts in a banana image was a good benchmark that worked for us. The few images that were not classified correctly had inappropriate background colors for our need such as yellow or brown. We plan to have a white background and good lighting so we are not worried about those failures as they do not apply in our case. We also acquired free rotten bananas from a store and now we are going to test our algorithm on those bananas. This next week, I want to finish our conveyor belt and decide whether to use AlexNet or not as we may not need it for our purposes. We are slightly behind track on the conveyor belt as I hoped to get it working this week but we are close and I plan to ramp up and ensure we have it working soon next week.
Team Status Report for 03/27/21
Our team made good progress this week on various elements of our project. We now have two masks for rotten and good bananas respectively that we will use to calculate the percentage of rotten parts in a banana. We also have started researching our secondary ML solution and looked at AlexNet as of now. We had a meaningful discussion with Prof. Tamal and our TA Uzair who gave us the great idea of using live feed or burst pictures instead of periodically taking pictures which we talked about more as a team and decided would be better for our timing concerns on our conveyor belt. We also know that building a conveyor belt is an important part of our testing and so we have started building out the conveyor belt. We made good progress today in the woodshop. We managed to find free wood pieces that worked for us and we have cut all the parts necessary for our conveyor belt, including the wooden base, the wooden side stands, the PVC rollers. Next, we have to hook up the motor to the PVC rollers with woodwork and we have a few ideas on how to do that. So we are on schedule for our project as of now. Kush is going to also work more on the Nano early this coming week and Ishita K. and S. should have a good rottenness detector using masks developed as well as started on the ML plan. Our conveyor belt should be ready by mid-coming week.
Ishita Kumar’s Status Report for 03/27/21
This week I met up with Ishita Sinha to work together on our rottenness detector. I improved the mask for detecting rotten bananas through the help of plotting rotten bananas on an HSV graph and then testing values on images of rotten bananas. My algorithm now has two separate masks one for rotten and for good parts of the banana and I worked to separate the two masks as before I just had a single mask for the whole banana. Ishita Sinha and I discussed ways to improve and implement our rottenness classifier and we decided using the areas found in our image segmentation masks would be a good start to calculating the percentages of rotten parts in fruit so we are implementing that. I also researched AlexNet and double-checked an online Kaggle dataset I found earlier to see if it would fit our needs and I believe it would as of now. I also worked with my team to build the conveyor belt this week and we made good progress with the woodwork and we will continue working on it early next week too. I am on schedule for my tasks.
Ishita Kumar’s Status Report for 03/13/21
I worked on improving my image segmentation algorithm this week. I noticed significant improvement from last week when using HSV color thresholding. This is because I decided to graph the HSV range in one banana image and then used information from that graph to decide my thresholds, instead of manually fixing the range through trial and error. This HSV color threshold does well on multiple fresh bananas too. I tried my algorithm on different images of bananas in different positions for testing it. I now need to work on segmenting rotten bananas as well. Ishita Sinha and I are going to merge our code and then work on classification. I am researching classifiers which I will discuss with Ishita Sinha. I will also work on the design document due next weekend and our team is going to meet to discuss this. We want to finish the classification code for rotten vs fresh by next week.
Ishita Kumar’s Status Report for 03/06/21
This week I talked with the team and decided on parts to order. We sent in an order for the Raspberry Pi camera and the Nano and received it already. I worked on image segmentation on a picture of a banana from google. I converted the image from rgb to hsv space and extracted colors within a certain range of yellow using a mask. I am going to continue working on improving the mask by using it on different pictures of bananas. I also worked on the design presentation slides. For next week, I am going to improve the segmentation algorithm and add to it to segment black spots on bananas as well. Then, I will try to merge my code with Ishita Sinha’s so we can do color analysis.
