Category: Sarah’s Status Report

This week, the team prepared for the final presentation, and I presented it. We met on Friday to embed the live stream onto the website and to have the RPi CV application run when the hardware and website are up. I created a video script for the CV application part of the demo and took images and videos of the CV application at work. I worked with Kanon to get the live stream into the website, and it was a simple embedding an HTML iframe tag and referencing the localhost the video could be browsed on to the bottom of the website. The team also completed the final poster, and Kanon and I connected our block diagrams to show the full integration of the CV application onto the website and through SMS.

I was able to catch up and complete the remaining tasks for the CV application and its integration to the whole greenhouse. All of our components are working, and we are ready for the public demo.

Next week, I hope to add the testing results, the changes on the schedule and design, and overview of the project and class into the final report. My team and I will also prepare for questions or demonstrations that may be requested in the public demo.

This week, the team worked on combining all the components together. I went to Hiroko’s place in the beginning of the week and recalibrated some of the HSV filters and edge detection to work with the background of the greenhouse and the UV lighting. Because some of the flowers that I brought from home were withering, I went to buy more flowers and plants to test under the greenhouse. I tested my growth stage classifier with the current pea shoots I have and with images of the pea shoots from sprouting to maturity. I also tested this component with the new flowers I bought to make sure that the flower and fruit detection were working for the growth stage classifier. For the disease and withering detection, I tested this on my peashoots, withering Pansy flower plants and the new flower plants I bought. For the stem detection, I tested this with the old and new flowers I have.

I am slightly behind on my tasks, as I need to work with Kanon this week to embed the live stream onto the website. I also would’ve liked get the CV application to automatically run when the RPi boots but I will work on that next week.

Before the final deadline for the poster, video, and report, I will be working on a script that turns on the CV application automatically when the RPi boots. Once I get those tasks done, I will try to recalibrate the disease finder, because the false positive and negative rates are too high as of now.

This week and the week of the Interim Demo, the team finished up their individual components and completed the Ethics Assignment and Discussion. I worked on recognition of darker spotted diseases on plants. I simulated this by coloring in some of the leaves with reddish/ blackish dots which is the most common pattern of plant disease. Because of the background noise, I implemented a more accurate edge detection method to white out the background, and since some of the disease detection got in the way by the fruits or flowers, I made sure to white out those parts of the image too and only analyze diseases on the leaves. I also implemented bending of the plants by measuring the midpoint of the pot, then connecting flowers to the midpoint to see how bent the stems are from the roots of the plant. I added the Twilio API messaging system to report growth status updates through SMS as well.

I am slightly behind, because I planned on integrating most of the components together by today, but I am having some issues detecting dark spotting diseases, so I will be working on correcting that today before I move on and will work on connecting the whole system tomorrow.

Next week, I plan to have most of my testing done so that I have some testing and metrics to show for the final presentation, I would like to work on setting up the RPi to run the CV automatically when the RPi boots. Since I am coming to Hiroko’s place on Monday, I will also integrate the RPi and CV application with the greenhouse, so hopefully we can get the live stream embedded onto the website.

This week, we all worked on our individual components to present on the Interim Demo. I finished my fruit/ flower detection and improved it so that it can find the most common types of flowers and fruits by working with the most common flower and fruit colors. I combined the fruit/ flower detection that I implemented with HSV Color Detection, and the pixel per metric function to make a growth classification system that will use these two parameters to notify the user on whether a new growth stage has been reached. The notifications are sent to the website and the user’s phone number through the Twilio API. I also made a program where the plant is separated from the background using Edge Detection, then the fruits or flowers are wiped out with HSV color detection to get the leaf alone. I am currently testing for the program to be able to detect white and discolored brown spotting, which is a common plant disease pattern, as well as withering. Further, I realized that I could be more specific with my testing metrics such as the accuracy of my pixel per metric when predicting the real height of the plant, so I will change some of my metrics to be more specific and critical of my application.

I am a bit behind schedule, as I would’ve liked to have my application run smoothly by Friday, but I still have to debug the disease detection and the Twilio API notification sending system. I will spend my weekend doing so and will have these functionalities prepared for the Interim Demo.

Next week, I hope to start on the plant vine and stem bending algorithm and test that out the following week. Based on the critiques from the Interim Demo and my own judgement of the quality of my CV application, I will change parameters and fine tune my application in the remaining weeks. I also plan on making stricter testing requirements and testing the CV application with these metrics in mind.

 

 

 

This week, we continued to work on our individual components, and we all planted the pea shoots. I tested whether night vision works in complete darkness, and it does so there is no extra LED lights needed in the dark. I’ve decided to turn off some CV analysis once the camera switches to night vision, because without the original colors of the image, we cannot perform proper analysis. Further, the plants grow very gradually day by day and assuming gardeners will be asleep at night, its unnecessary for the constant analysis. I was able to test my pea shoots that already sprouted, and using the pixel per metric method, I was able to classify the pea shoots as sprouting. I will need to adjust the camera and planting tray position to get a perfect sideview of the pea shoots for more accurate plant height calculation. Currently, I am working on making each leaf and flower an object and going through the list of objects to distinguish the difference and determine whether they are healthy, diseased, or withered with some edge detection and another layer of HSV Color Detection.

I am a bit behind schedule, as I wanted to test some pea shoots for defects, but I can’t do that until the pea shoots grow a little more so by next week they should be good to detect defects and I still need to fix some issues with my code. I changed my application to detect defects in the young plant, flowering, and harvesting stage, as plants tend to be diseased or withered around this time.

Next week, I will be working on sending notifications with the Twilio API to the website and SMS, as this is a part of the integration of the CV analysis to the website. Since the growth stage classifier is almost completed, I will test whether the user receives the correct notification about what growth stage the plant is in, and if it sends the correct amount of notifications. I also hope to have the leaf and flower detection, and defect detection completed before Interim Demos.

.

This week, the team and I continued to work on our individualized parts. I was able to complete the live streaming script for 24/7 monitoring of the greenhouse, and I am hoping to link that to our website once the RPi is sent to Kanon and Hiroko.  Instead of doing HSV Color Detection and Edge Detection on online images of pea shoots, I was able to do that with images through my RPi of some of the succulents I have at home so that the CV is applied on realistic greenhouse/outdoor lighting. Currently, I am working on getting the CV to distinguish leaves and flowers, and I’ve completed my pixel per metric by measuring the bounding box that outlines the stem and top of the plant in the image to the real size of the succulent that I was testing. I figured out a way to work with the RPi without using a monitor or keyboard and instead using a VNC viewer which connects to my RPi through Wifi and displays the RPi OS right to my computer, which allows me to test my CV analysis through real time videos from the RPi instead of through RPi images . I also planted some of the pea shoots so that some can sprout by next week and I can properly test my growth stage categorization algorithm on pea shoots rather than the succulents.

I am slightly behind my tasks, as I wanted to test the night vision and get the flower and leaf recognition implemented, but I completed the growth stage pixel per metric algorithm and refined my HSV Color Detection and Edge Detection. My RPi stopped working in the beginning of the week, so I had to borrow my friend’s RPi until my new one came through.

Next week, I hope to be able to test my growth stage classifier with the sprouting pea shoots. I would like to finish my implementation on leaf and flower recognition, so that I can have the minimum to test the defect detection which would use another layer of HSV Color Detection on the parts of the leaf and flower. I will also make sure that the 24/7 monitoring is applicable during night.

.

For this week, the team prepared for the design review presentation and worked on our individual components.

With the equipment all here, I was able to connect the IR-Cut camera to the Raspberry Pi, and create a webcam connection to the laptop with the MicroSD card. I am currently using a random plant in my house to find the right HSV values to extract and the best lighting for CV analysis. I was also following up on tutorials on how to do CV analysis of live streaming images.

I spent time on writing the project management and summary components of our design review documentation, and will be working on the design trade studies and systems review portion of the document.

I am a bit behind my CV analysis implementation, as I wanted to have some sort of analysis implemented through the RPi camera, but the RPi connection took a bit longer than I thought and I had midterms this week. As for the group as a whole, we are on time with our deliverables and will be meeting on Monday to finalize the design document.

By next week, I hope to have the growth stage classifier and half of the disease detection implemented with the extra time I have from no homework in other classes for this upcoming week. I will also start growing one of the pea shoots because I need some testing materials for the growth stages.

This week, my team members and I researched and designed the components of the projects individually assigned to us. In particular, I looked into several tutorials and found built in functions in the OpenCV module that would help with differentiating the plant from the rest of the video for CV analysis to be done. In speciic, I looked into the pixel per metric technique to figure out the real life size of objects through the camera, HSV color detection and edge detection to detect withering, disease, bending, and flowers/ fruits. Using a pea shoot image I found online, I found certain color hue, saturation, and value that would separate the bundle of pea shoots from the rest of the objects in the captured image such as the soil, the background, and the planting tray. Afterwards, I implemented some edge detection to get a clear outline of the shapes of the stems and leaves. My teammates and I also ordered our materials on Monday, and I have received all the hardware I need such as the RPi, RPi power adapter, RF transmitters and receivers, and planting material.

My progress is on schedule, as my team just needs to make some final adjustments on the design review presentations after receiving feedback, and we are all started on the implementation process of each of our tasks. I learned a lot more about OpenCV this week and have a much clearer idea of how I can use the built in functions to cater to image and video analysis of plants.

By next week, I hope to connect my camera to my RPi, and work with the RPi camera for my CV application. I would like to figure out the HSV parameters out of the images that the camera provides and I will most likely use some plants I have at home to figure out the basics to HSV Color and Edge detection specific the camera and lighting at my place. I would like to accomplish the pixel by metric technique and the flower distinguishing algorithm by the end of next week.

While the presentations were going on and Hiroko presented, my teammates and I decided to study our individual parts of the project. I downloaded OpenCV on my computer and tested some of its functionality . I also looked into the best cameras and boards to maintain a 24/7 live stream. With the OpenCV, I looked into edge detection, how the module recognizes and categorizes certain objects, and how to separate colors, as all these aspects will be key to the defect and growth stage detection. I also tried to figure out how OpenCV categorizes shapes and the size of objects.

The team as a whole is on task, and we will be meeting next Monday to gather what we’ve been working on, and the equipment and materials we need to order. On my part, I would’ve liked to learn more this week about applying CV on plants in specific and get a rough idea of how I will measure some components such as how to figure out the degree of bending of a stem, but hopefully more tutorials will help me understand my part of the project.

I am hoping that we know what to buy and let Abha or other TAs know by Monday and Wednesday the latest. I am also hoping that we can rent some boards like a Raspberry Pi, as that would help our budget. On my part, I am hoping to look into more tutorials and edge detection/ color classification of OpenCV throughout the weekend.