Category: Team Status Report

This week, the team worked on putting all the components together. Kanon and  Hiroko worked on the communications between the website and the hardware, and recorded the test results to verify metrics have been met. Sarah came over to the greenhouse and recalibrated the CV application parameters based on the different lighting and picture quality from placing the plants in the greenhouse.

Some risks that we encountered this week were making the soil moisture sensors waterproof. Hiroko was able to find some heat shrinks in the ECE department and use that for the soil moisture. Sarah currently has a large false positive and negative rate on the dark spotting disease detection, so she is going to cover the area surrounding the plant with white paper to get more accurate testing results.

There were slight software infrastructure changes made to the CV Application. Instead of finding a stem, Sarah decided to measure bending by measuring the angle between the Y-axis and the line between the flower to the plant center, since this was a much more efficient and accurate way to detect stem bending. She has changed that in her block diagram.

We are on task so far, and Sarah and Kanon’s collaboration to embed the live stream on the website is the only thing pushed to this upcoming week.

Last week and this week, we prepared for interim demo, ethics assignment, and also worked on our individual components. Hiroko had some trouble this week because the junction box for assembling hardware did not arrive on time. This delays Hiroko&Kanon’s testing process, which they will immediately start working on starting this week. Kanon refined the website UI based on the surveys she took and improved the backend algorithm so that website and hardwares can communicate more smoothly. Sarah worked on refining her image processing algorithm for the recognition of the diseases. She also implemented Twilio API to notify the user about the plant growth/change.

As a team, we seem to be slightly behind the schedule because of the delay in arrival of the junction box, testing process, and some issues with image processing. We will try to catch up next week by focusing on the testing process. Also, because everyone from the team will be in Pittsburgh, the integration of the entire system should be more smooth.

This week, while we worked on our individual components, we also began to connect those components together. Sarah was able to embed the video to a simple HTML site running on local host, so we are expecting the transition from this to the actual website will be smooth. Hiroko and Kanon worked on the connection of the ESP32 to AWS, and were successfully able to send and receive data. All of us also planted some pea shoots for testing. Sarah was able to test her growth classifier implemented by detecting the sprouting of the pea shoots. Hiroko and Kanon are now able to test the sensors and make adjustments for better pea shoot growth.

Some risks that we need to look out for on the hardware side is to setup the wires and boards safely so that no water, moisture, or biomaterial will touch it. Depending on how the setup goes, we may need to make some containers and use zip ties to tie the hardware to certain parts of the greenhouse. With the TechSpark lab equipment and materials nearby Hiroko and Kanon, we can create containers if necessary. For the computer vision, the background of the greenhouse may be in the way of proper analysis, so to prevent that we will have a monochrome background to mitigate such issues, and Sarah is currently working with a box as her background for the CV analysis. The camera may also be difficult to adjust once it is placed in the greenhouse, so we are considering a RPi camera module stand and mount if we find issues fixing the camera onto the greenhouse.

Sarah had to adjust some parts of the CV implementation systems. Since the real colors of the greenhouse are washed out in night vision, we’ve decided to only have CV analysis going on in daylight when the IR filter is switched back on. We found that it is unnecessary to have the CV running when the users are asleep and when plant growth is very gradual. Further, the leaf and flower detection along with the defect detection will only be applied from the young plant stage to the harvest stage and not in germination, since most defects occur during those growth stages. Kanon had to calibrate some values that the soil moisture sensors read to percentages for user readability, and with the deployment and user testing coming up, she may have to adjust some of the UI. On the hardware side, Hiroko needs to adjust the watering tube so that water is distributed more evenly and adjustments like the making a container may be needed when the system is integrated to the greenhouse this upcoming week.

The schedule is mostly the same, but instead of shipping the RPi and the camera, Sarah has decided to bring the RPi and camera in May. Full integration of the greenhouse is therefore delayed by a week, but that gives us more time to work on our individual components and Sarah setting up the CV equipment herself will save more time for the integration process.

While most of our work was still individual, starting last week, we started to combine our components little by little.

Last week, Hiroko and Kanon worked together to connect ESP32 to AWS components. We are now able to send data from ESP32 by AWS Lambda functions, store the data within AWS DynamoDB, fetch the data and print them on website, and send back data to ESP32 using curl command via our backend.

Hiroko also worked on setting up the hardware devices by soldering them. She also got a relay controlled outlet box to improve our design aesthetic and also user safety. She is now working on connecting sensors properly and calibrate those values.

Sarah had a difficult time because her Raspberry Pi broke last week. However, by borrowing her friend’s RPi, she was able to continue working. She can now show the 24/7 live stream directly on the website frontend. She also tested some openCV functions on real plants and completed the growth stage classifier.

Because there was some unexpected problem such as RPi breaking, we are glad that we made a week of slack within our schedule. There will be no change on the update for our schedule, and we will be still working individually next week to refine each of our components.

This week, we all worked on our individual design block diagrams for the design review presentation, looked into more risk factors, and began implementing our components. We also ordered all the materials we need and we have received most of them. We made a list of bills and materials and are working on the design documentation report as well.

Hiroko looked into the sensors she will be working with and ordered all the materials that she needed and picked up a few from Home Depot. Then, she made a visual design of how the greenhouse will look like and created the hardware block diagram which specified all the relays and feedback loops in the system. She also signed up for access to TechSpark in order to solder the sensors to the ESP32 , and hopes to receive some help from TAs to properly soldering the sensors. Kanon created the website login and registration as well as the main page where there is a toggle to change temperature, a switch for turning on/ off the light, a. soil moisture modifier, and a section for putting the live stream video later on.  She also found Twilio, an API aids in notifications and alerts to users, which would help our project significantly. She updated her block diagram with the Twilio API included. Sarah researched the OpenCV modules more, and layed out the algorithms and components she would need to properly implement a CV application for plants. In specific, she figured out HSV Color detection and edge detection for images. She also received her materials and is setting up the hardware to do proper CV analysis.

Some risks we are looking into is receiving the wrong data from the database, and to mitigate this issue we are thinking about either regaining the data before outputting the value to the website or to notify the website if drastic changes take place. We are also hoping that the night vision in the IR-Cut cameras will work properly, but in the case that it does not we will need to reconfigure the LED lights to be on a certain brightness for night vision to work. After testing the OpenCV module, we found it very important that the subject we are analyzing contrasts with the background or the unnecessary components of an image, so we are looking into making a monochrome backdrop in the greenhouse that provides the best contrast. Any unexpected issues that we may come across with the hardware, we have decided that our system will be able to detect and notify the user about it.

We updated our schedule a bit to figure out when to start planting the pea shoots and which tests we will be conducting on which days. We’ve decided that we would plant the pea shoots a week before testing, as it takes around 7 days for pea shoots to sprout and grow. Below are also some images of visible progress.

 

Web site progress:

This week, we focused on the refinement of our design so that we can get prepared for the design review presentation for next next week. As we focused on each of our parts, we hope to talk about our ideas on Monday/Wednesday to put them all together.

Hiroko worked on her hardware component, coming up with a block diagram and shopping list which helped us go over the scope of the project and organize what we really need.

Sarah has downloaded OpenCV and started to go through some tutorials and learn basic algorithms such as edge and color detection.  She will continue learning algorithms and OpenCV functionalities next week to get more specific ideas for the project.

Kanon has implemented OAuth to web application and will work more on the design and web development next week. She will also look into AWS database installation and pricing.

Next week, we will start ordering the parts we need and ask TA/Professor about borrowing Raspberry PI boards so that we can get started on building up hardware systems.