Ashley’s Status Report for 9/28

This week I worked with my teammates on the design presentation and we made some changes to the design. The hardware part will mostly remain the same, so I continued with my research on the hardware components I will be working on. We finalized which model we wanted to use, and placed the orders for the ultrasonic distance sensor and the Arducam camera.

I also found a software called Tinkercad that allows you to simulate circuits. I saw that it includes the exact ultrasonic sensor that I wanted to use, and it allows you to write code. I played around with it a little and hopefully this will help me test and verify that it works before I start working with the actual components. Tinkercad lets me do something like this:

I am a little bit behind schedule, due to many overlapping deadlines and upcoming exams. I have done research on how the components work and what pieces of code I would have to write, but I don’t have any solid components yet. In the next week, since I expect to have a bit more time, I hope have a more solid design of the hardware and start setting them up once the orders arrive.

Mandy’s Status Report for 9/28

This week, I started by preparing for the design presentation. Since we made a lot of changes to our project, we had to add these changes to the presentation. In addition, since I am presenting for  the design presentation, I spent the majority of our time writing out a basic script of things that I need to say, and spent time rehearsing it.

Since we also added a new section to our project, I spent some time researching equipment and materials that we could use, including the type of display. Originally, I thought about using a raspberry pi, since raspberry pis have more compatible display screens. However, I decided on using an Arduino GIGA display shield, as it is a display that is compatible with Arduino, which we were already using to control the door.  In addition, it is a touch screen display, which can allow users to interact with it, making it feel more like a game.

I am slightly behind on my schedule, since we just decided to add an extra part to our project a few days ago. I have been able to do some basic research and brainstorming about it, but in the following week I must finalize the equipment that I need to use, and place orders for it within the first few days of the week.

In the next week, I plan to place orders for the items that I need, draw out specific diagrams for what the display would look like and what it kinds of statics need to be shown. I will also begin the coding for the assignment.

https://store.arduino.cc/collections/displays/products/giga-display-shield

Justin’s Status Report for 9/28

This week I was able to get YOLOv7 up and running on my computer to test its performance on some sample images. This was purely to gauge the model out of the box, and as expected, the classification was not very good on recyclable materials.

This was expected, and confirms our need to find a suitable dataset of recyclable materials to fine tune the model. Our original plan was to train our model on some labeled recyclable materials (ie. aluminum can, plastic bottle), and have some sort of lookup table for each classification to determine if the item is recyclable. This could potentially lead to cases where an item isn’t recognized with high confidence, and we would have to make a decision on what to do in such a case. An alternative we discussed was customizing our dataset so items are labeled as “recyclable” or “non-recyclable”, but currently I am leaning towards the original idea, with unrecognized items being simply classified as non-recyclable.

Aside from CV work, most of my time was spent on the design presentation, and working out the details of our new game system. The game system won’t interact directly with the CV, but adding a new system to the project felt like something for the whole group to discuss.

Progress has been good so far. I will be (temporarily) leaving the country in the middle of next week. I should (and plan to), work on assignments while I am gone, and luckily I should be able to do CV work remotely, but between being gone, and making sure that I’m keeping up with other classes. I’m anticipating that I may not get as much work done next week. I’m hoping to find a suitable dataset and figure out how to train the existing model on this dataset. One that I have found is this: https://www.kaggle.com/datasets/arkadiyhacks/drinking-waste-classification , but plenty more are available, and I’d like to find a dataset that may include items that are often thought to be recyclable but actually aren’t (ie. plastic bags). If the Jetson camera arrives before I leave, I also plan to hook it up to get the Jetson to capture images.

I will make sure to communicate with my group if I have any issues.

Team Status Report for 9/28

As we are finalizing our design, the most significant risk we are facing is figuring out how our system will handle the items that the model does not recognize. If the model is unsure about the item and cannot classify it, we weren’t sure if we should make the user retake the picture of the item for a more accurate result, or if we should have a rejecting system that will not place the item into any of the bins. We decided that we will place the item into the trash bin because it is safer to have something mislabeled as trash rather than contaminating the recycling bin with non-recyclable items.

Since we have switched from four categories of recycling, to just recycling and trash, we decided that having a sliding gantry was no longer necessary, and chose to go with just a single door that tilts right or left depending on what the item is sorted as. This change allows us to minimize the amount of time it takes for items to be placed into their respective categories, and reduces possible mistakes that could happen when placing the item. The cost of the change would also decrease the overall budget, as we would not have to purchase a gantry.

We are also planning to add a new system to our project, which is a game where the bin tracks the number of items recycled in a week, and compares it to the past week, as well as a running average of items recycled per week. This tracking system will work similarly to the setting in iOS that tracks weekly screen time, and serves to track overall recycling “performance” for the household. The statistics will be accessible through a display on the bin that connects to our Arduino. Users will be able to control the display with buttons to view different statistics (ie. this week’s total, average recycling per week, % of items in bins that were recycled).

Some pictures of running YOLOv7 off the shelf:

It works quite well for some images/objects, like this image provided in the repository

But not so well in detecting recyclable materials. Fine-tuning will be needed.

Part A-Ashley Ryu

With respect to considerations of public health, safety, and welfare, our project aims to address these aspects by encouraging better recycling habits. Public health is addressed by reducing waste and limiting exposure to hazardous substances, which can be caused by improper management of trash. Safety is addressed by educating users about proper recycling, and reducing the risk of people handling hazardous materials improperly. For example, batteries should be handled separately instead of going into the trash/recycling bin, because they contain toxic chemicals that may cause fire. Our project aims to identify these hazardous materials in addition to trash vs. recyclable. Lastly, we address welfare by promoting a sense of responsibility and awareness of environmental issues through better recycling practices.

Part B- Mandy Hu:

We intend for our project to impact society by making recycling a more prominent part of people’s daily lives. When doing research for our project, we compared recycling in America, where most people don’t care about it too much, to South Korea, where sorting recycling is normalized, and people who don’t recycle are looked down upon by society. By keeping track of recycling statistics, people will become more aware of how much trash they are consuming on a daily basis, and become more diligent about what products they are purchasing. In addition, by simplifying the recycling process and making it quick and easy, we want to make it so that even people who are busy don’t have an excuse to not recycle.

Part C – Justin Wang:

Since our project aims to be a household product, it should not be incredibly expensive. To that end, we are using mostly parts that can be purchased off-the-shelf and are easily accessible. The custom parts of the project are mostly software and system design, and thus can easily be replicated. The design aims to minimize complexity and parts required, which should reduce purchasing and production cost and make assembly easier. The bulk of the price will come from the Jetson, which does cost a significant amount, but the simplicity of the rest of the design should ensure a scalable production process that would make EcoSort easily accessible to those who want it.

Ashley’s Status Report for 9/21

This week, I met with the team on Friday to reflect on our proposal and project design. I did some research on the parts that we need to buy and the software that we will be using. I looked into the ultrasonic sensors that could be used to detect the object, and read through some Arduino code tutorials. I found some parts online that we could buy for the mechanical part of our system. I am on schedule for now, and for next week, I will continue with my research and hopefully finalize the items that we will be buying. I will also work with my teammates on the design presentation.

Team Status Report for 9/21

A potential issue that was raised was the accuracy of a YOLO model, which tends to be used for applications where speed is more important than accuracy. We are currently looking into the possibility of more powerful object detection models, but are leaning towards a more recent version of YOLO, since it is easier to set up and run, and its wide use in past 18500 projects means that it is proven to work. Aside from the CV, a potential area of concern is the system to drop items into the bins. We’ve come up with an arrangement which should simplify the system as much as possible, with a linear gantry only requiring control to a stepper motor which will “select” a bin, and a servo to drop it in. In this way we avoid complex robot kinematics. If we end up switching to a single-stream system (more on that in the next paragraph), we will have only two categories, meaning the system could be further simplified.

Based on the feedback we received from our proposal presentation, we are considering changing our design to sorting into two bins, recyclable vs. trash, instead of four. Pittsburgh (and many other American cities) use single-stream recycling, which means all recyclable materials are collected together then sorted at the processing plant. Some places may collect sorted materials (ie. CMU has individual bins for different categories of recycling), but some places may simply collect our sorted materials and dump them together. Sorting into two bins will likely not have much impact on our computer vision system, which will still recognize and classify objects as we planned, the only difference being that classified objects will then be grouped based on whether or not they are recyclable, instead of the specific recyclable material (ie. water bottle → recyclable, rather than water bottle → plastic). There isn’t too much change on the hardware side, the system design will remain the same, only we have fewer categories. The biggest changes are in the mechanical design, primarily in part selection and physical size. With half as many categories, our bin can be physically smaller, and we can use a smaller gantry for item placing.

Since this change doesn’t impact the overall design of our project, we don’t anticipate our schedule to be impacted too much. This could have the potential to push back the hardware assembly part of the project, since we may be delayed in ordering the gantry, but we still have to go through the design process before starting assembly, so the delay should be small, if there is one.

Justin’s Status Report for 9/21

Most of my time this week was spent preparing for the proposal presentation, with some research in the CV system, as well as looking into parts we may want to order. We received some feedback from the presentation about the YOLOv4 tiny model that we planned to use, with a suggestion that we could try a more recent YOLO version, or even a different model entirely that is more accurate (ie. Faster R-CNN). Since YOLO is rather easy to set up, I am currently working on setting up YOLOv7 to try training on a custom dataset. Our order for a Jetson from the 18500 inventory has been approved, so hopefully I can try deploying YOLO on it soon. I have found multiple datasets of recyclable materials (ie. https://www.kaggle.com/datasets/alistairking/recyclable-and-household-waste-classification), so we have plenty of training data available.
I also looked into serial communication between the Arduino and Jetson. Both devices have serial communication pins that can be used to talk with each other. It is also possible to connect the two with a micro usb cable, which will likely be our approach. Communication between the two devices is an important part of meeting our project’s use case, so it’s good to see that this is possible and has been done before.

Mandy’s Status Report for 9/21

This week, I did research on the components that we are looking to purchase for our project. I double checked that the components we are looking into are able to fulfill our requirements and are compatible with each other. I also spent time researching the current state of recycling in America, to determine whether or not we should change the scope of our project. I also started working on the design proposal. I am on time with our schedule, with the design proposal still in progress. By next week, I hope to finish the design proposal, and send in requests for all of the materials that we need for our project.

Introduction and Project Summary

An important step in the recycling process is sorting items based on their material. Miscategorized materials can lead to recycling contamination, which can degrade materials, resulting in higher costs, and cause harm to employees responsible for sorting. Currently, places like Pittsburgh use single-stream recycling, where all recycleable materials are collected and processed together. Our project EcoSort aims to make sorting your recycling at home easier, with bin that automatically sorts your recycling using computer vision. EcoSort will place the item in one of four categories: plastic, metal, glass, and paper, and will do so without any additional user input beyond placing the item in the bin. We aim to both reduce micategorization and promote home recycling by making it easier for the average person.