Posted on

Team Status Report 3/8

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The most significant risk to our project is that the suction-based end effector won’t be able to pick up different types of trash items. We’re managing this risk by focusing a considerable amount of time on the development of the end effector mechanism. In the case that the suction with the vacuum pump is ineffective, we plan to pivot to a claw like end effector mechanism for picking up the pieces of trash.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

There are no current changes made to the existing design of the system.

Provide an updated schedule if changes have occurred.

There are no schedule changes as of now.

Part A was written by …., Part B was written by Ethan, and Part C was written by Alejandro.

Part A:
with consideration of global factors. Global factors are world-wide contexts and factors, rather than only local ones. They do not necessarily represent geographic concerns. Global factors do not need to concern every single person in the entire world. Rather, these factors affect people outside of Pittsburgh, or those who are not in an academic environment, or those who are not technologically savvy, etc.

Currently, the issue of proper waste processing is one that many countries, including the US, are struggling with. Many countries do not have the infrastructure to pay for recycling facilities, as it requires a lot of manpower, and the resulting materials are not very profitable. Additionally, a lot of our waste is shipped to other countries in order to prevent it from piling up here, however this means that other countries are burdened with our trash. SortBot could help reduce the amount of trash by separating out the useful materials, at a much lower cost compared human workers. This could potentially help other countries without the funds for recycling management to do so.

Part B:

In the United States, the common practice is to dispose of all types of municipal solid waste in single bin, prioritizing convenience over environmental concern. SortBot is set out to challenge this norm, one that focuses on ease and efficiency, by introducing an autonomous system that streamlines waste separation without requiring a behavioral change. By shifting the responsibility of sorting waste away from the person throwing away the trash, SortBot can work within the existing American cultural norms. This ensures that materials can be property sorted without requiring individuals to change their habits. While this is not the best solution for this issue, it is one of the most appropriate for current American culture.

Part C:

Our product solution will meet the need of the consideration of environmental factors by providing a streamlined method to making the environment cleaner. Utilizing advanced computer vision and machine learning algorithms, our robot has the ability to identify and differentiate between different types of trash and can thus correctly categorize the trash types and sort them into their respective bins. The system can aid in the sorting of trash to aid in the recycling of these items. Normally, all recycling items mixed with trash would normally end up in landfills and be lost in terms of the value that they could provide when recycled, but now with our product solution, we can recover these items and help in reducing the environmental strain of waste disposal.

Posted on

Alejandro’s Status Report for 3/1

I spent the majority of the week working on the assembly of the x-y axis of the gantry system. The assembly was time-consuming and laborious. The assembly requires a few more steps that will take some considerable amount of time. I will work on finishing the assembly the week I get back from Spring break. My progress is slightly behind schedule. In this case, I will prioritize the assembly of the gantry’s xy axis in the following week of work. Additionally, I was experimenting with the use of the canvas drawing onto the camera feed but I will need more time to solidify the feature. I’m also running a bit behind with the canvas drawing feature but I’ll have that done promptly after assembling the gantry. So next week I intend on completing the gantry xy axis and finishing the implementation of the bounding box HTML feature of mine.

Posted on

Ethan’s Status Report for 3/1

The majority of effort this week was spent on finding bugs in our YOLO codebase that caused the results from last week to look really poor. I discovered it was the loss function. Previously I was using a more naive approach that combined a weighted mean-square loss for the bounding boxes with a weighted cross-entropy loss for classification. After reading a couple of Medium articles about YOLO, I realized that I was implementing an entirely different loss function. Once I fixed that I also a little bit more training infrastructure that would hopefully making training analysis easier: curve plotting for each loss. By monitoring loss, I can identify parts of the model that would need tuning in future runs.

Next week, I plan on getting detection working on toy images.

Currently I am on schedule.

Posted on

Teddy’s Status Report for 3/1

Over the last two weeks, I worked on constructing the gantry as well as the frame that holds it. I was able to construct the 4xidraw part of the gantry since all of our parts had finished arriving. I was then able to print, assemble, and attach the z-axis to the gantry. Currently, the parts required for the z-axis movement are assembled, but the end-effector is not attached, since we are still waiting for the vacuum pump and the relays to arrive. There is an issue in that the z-axis can’t extend very far away from the gantry, as the weight causes the rods to sag. In order to remedy this, I’m planning on attaching a bar horizontally onto the part where the z-axis attaches to the rest of the gantry, which will rest on two other bars so that the front does not sag. A picture of the current gantry is shown below.

I am a little behind schedule, partly due to delays in the shipping of the gantry parts as well as the end-effector parts. I plan to fix the sagging issue on the gantry next week, as well as hopefully get the electronics and the code for the 4xidraw working.

Posted on

Alejandro’s Status Report for 2/22

I successfully established the WebSocket connection between the Jetson Orin Nano and my laptop’s localhost server. I then configured my server.js to stream the video served over the WebSocket connection to my web app. Additionally, I had to change the index.html code to properly display the video being streamed. After several hours debugging, I finally got the webapp to stream video. Now, the web app displays the video of the camera that’s connected to the Jetson Orin Nano. Here’s an image of the video feed of me being displayed on the webapp. 

I am currently slightly behind in that my team had planned to assemble the xy axis of the gantry system this week; however, my team ordered only two rods of 3 ft when we expected two rods of 6ft each. The other two 3ft steel rods should arrive next week. In order to make up for this delay, I’ll take a day sometime next week to assemble the xy axis of the gantry system.

Next week, I need to also implement box shading onto the HTML of the webapp to display identified objects on the camera feed. And as previously stated, I will also assemble the gantry system. At the very least, I’ll assemble half of it in the case that the other two steel rods do not arrive in time.

Bytes Sent over Websocket Between Jetson and Server

 

 

Posted on

Ethan’s Status Report for 2/22

This week, I was able to finish the training infrastructure to train the YOLOv8 OBB model. Now that I am able to train the model, I need to employ some sort of verification strategy determine that the model was implemented correctly before I do full batch training. I decided on training the model one single basic image (I am defining basic as an image where the object is close up and on top of a distinct background). After training on this image for a significant number of epochs, I found that the detected bounding box was completely off. Currently, I believe that something went wrong with the model’s OBB detection head and spent a majority time this week trying to verify this assumption.

Next week, I plan on getting detection working on this toy image and hopefully training using the entire dataset and analyzing the results from there.

Currently, I am on schedule.

Posted on

Team Status Report for 2/22

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Currently, the most significant risk to the success of the project is the end-effector design. It has to be able to adapt to a wide range of surfaces, and there is a decent change that our current design might not be enough to achieve that. We recently decided that we would be adding a vacuum pump to the end-effector in order to create a better suction for items that are porous, have irregular surfaces, and/or heavy. If this doesn’t work, we will pivot to a claw-like end effector.

Another risk to our design is our current depth estimator. Right now, we plan to use a stereo camera in order to obtain the height of the objects so that the end-effector knows how far down it should travel. If there are problems with the resolution or the reliability of the depth reading that we get from this camera, it would prevent the gantry from moving to the correct location. The current contingency plan is to add a pressure sensor onto the end-effector if the readings end up being too coarse or unreliable.

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

We’ve added a vacuum pump to the end effector, as from some initial testing the suction cup end effector is not of adequate consistency when it comes to picking up some basic trash items. This does not incur any costs, except to our budget.

Provide an updated schedule if changes have occurred. 

No schedule changes have been made.

Posted on

Teddy’s Status Report for 2/22

The bulk of the time this week was spent designing parts for the z-axis movement/end effector as well as working on making a functioning conveyor belt. I’ve created CAD models for the rack and pinion movement, and have started assembly of the parts. I received multiple parts that had been ordered, including the suction-cup attachment for the end effector, and after some testing, decided that it would be necessary to actively pull a vacuum instead of having just passive suction. After some research, I was able to find a way to pull a vacuum (with minimal noise so it doesn’t interfere with other demos) and have ordered the parts for the vacuum attachment. I’m currently working on designing the 3d printed parts that will attach the vacuum and the end effector to the rack and pinion system. Additionally, I was able to construct a frame for the conveyor belt out of 3d printed parts and aluminum (image of frame is shown below). Currently working on designing a basic belt that will be the surface that the trash will rest on.

I am currently a little behind schedule, as the switch to an actively-pulled vacuum requires additional design work. I’m planning on finalizing the design of the z-axis/end effector and working on the Arduino code for the gantry xy movement for the over the next week.

Posted on

Teddy’s Status Report for 2/15

This week, I started the design of the z-axis movement and the vacuum end effector. I did a bit of research on methods of vertical linear motion, and settled on a rack-and-pinion based system driven by a stepper motor. I’ve started to create and assemble the CAD files which will be printed out when the design is fully fleshed out. The relevant parts for the end-effector have also been ordered, and added to the BOM in a new section. I spent a few hours working with the stereo camera, and discovered that it has its own python wrapper which should hopefully integrate well into our other code.

Additionally, I printed out all of the 3d printable parts for the 4xidraw gantry for the xy movement. 

Currently, I am a little bit behind schedule, as I expected the parts we ordered to come in earlier so I could start designing around them.

Posted on

Alejandro’s Status Report for 2/15

This week I spent time investigating web socket video streaming methods for a web app on a local host. I realized that Django would be too slow as it isn’t optimized for real-time streaming purposes as it is mainly meant for handling database/multi-user applications. Nonetheless, I decided to shift the framework to Node.js since it’ll provide lower latency in video streaming. I worked on writing the JavaScript file to set up the server backend. Additionally, I rewrote the index.html file to display counters of the types of trash collected, a camera feed box, and information on the types of trash rules we’ll be following. I also, reinstalled the OS on the Jetson so that my team and I would have access to it. I also setup RealVNC viewer so that I could use the Jetson without connecting it to a monitor. To make this work without RealVNC, we may need to order a Vesa cable.

I tried streaming an mp4 video but encountered some issues on the front end. I’m going to continue debugging this so as to be on track with the video streaming component of the web app for next week.

Additionally, I was supposed to assemble the XY axis of the gantry system this week but parts are taking longer than expected to arrive. Thus, I will continue as planned and take a day in the upcoming week to assemble the XY axis of the gantry system.