Posted on

Ting’s Status Report for 3/11/23

This week we got the code for YOLO to sucessfully run through, and we were able to start training. This is a great development for us, as we had been working to decipher the file structure to get the code to run for a couple of weeks. It trained very slowly though, since it was running on CPU, with 1 epoch taking almost 30 minutes. We will use the GPU on the ECE machines to run the code. Vasudha and I met with Prof. Mukherjee on 3/2 to get FastX set up to be able to do that. If that does not work, I will be using my leftover GCP credits to train. We are on track in terms of the software, and the next step after break is to start mass training, getting in as many epochs as we can with whatever GPU limits we have.

We further improved on the mechanical design of our structure, discussing with the professor and TA about a wooden frame that would encase the bin. As we wrote the design document, we decided that we will have 2 designs, one of which is backup. If the wooden frame design is too hard to woodwork, we will have the lid be overhanging the rest of the structure, which serves as handles on the side. We spent most of the week working on the design document. We are slightly behind in the mechanical aspect. Hopefully our materials that we ordered will arrive by the time we’re back from break, and we’ll start construction when the team is back from break. New tools needed for us is using the GPU on ECE machines, and learning how to run our python jupyter notebook on it. 

Posted on

Team status report for 3/11/23

Great news for us during the last week before spring break: the model could finally run through training! On Friday, Ting and Vasudha met with Tamal to set up the HH 1305 machines environment so we could run YOLO on GPU, which would make training a lot faster. Coming back from spring break, we will run through training with 150 epochs (or more) and start fine tuning with more datasets that we have collected.

We have also decided to switch from Jetson Nano to Jetson Xavier NX after studying their capabilities. We determined that Xavier’s computational power, measured in FPS (frames per second) for GPU , is more suitable for running a CV model as complicated as YOLO.

We have also finalized our mechanics design (mostly the material and structure of the lid and the swinging door) after discussion with the staff meeting. Besides that, most of our work is devoted to finishing the design report. We have pinned down details of the core components mentioned in the design presentation and we have done more research to justify our design decisions. Overall, it was an all-inclusive overview of our design and experimenting process over the first half of this semester.

Our biggest risk is once again the mechanical construction of the lid, once again due to our lack of knowledge in the area. To mitigate the risk, we have spent 3 hours a week researching and redesigning to improve our design, and this past week, after talking to the TA and Professor, we once again updated our design, this time constructing a wooden lid frame supported by four legs that the trash can can be easily removed from for emptying. Here is a picture of our latest design https://drive.google.com/file/d/1kVl_is8CKyrQoq8ftapcW93fXAPjJfYD/view?usp=drivesdk. 

As contingency plans, we have kept several backup design plans that can be adapted if our main bin design fails. Regarding construction itself, we found that the techspark workers can possibly help us out with cutting the materials we need, lowering the overall risk as well. With the success of our model being able to run through training, we are on track in terms of software, but still a little behind mechanically due to the materials not yet arriving, as well as our recent changes in design. 

Another risk is not being able to get the code running on the 1305 machines. We mitigate this by setting up GCP credits. 

Next week our goal is to train as many epochs as we can on the 1305 machines. We also plan to start constructing our structure. 

 

Q: As you’ve now established a set of subsystems necessary to implement your project, what new tools have your team determined will be necessary for you to learn to be able to accomplish these tasks?

  1. Right after the break, we will start setting up Jetson and deploy our detection & classification code to it. So working with Jetson and the CIS camera attached to it will be a major task.
  2. We will also migrate the ML model from Google Colab to run on the HH 1305 ECE machines. We will use FastX as our new tool. 
  3. Although not immediately, after integration with Jetson is on track, we will also start some mechanical work such as laser cutting to build the mechanics section of Dr. Green.
Posted on

Ting’s Status Report for 2/25

This week i presented on the design presentation. one question that was raised was about the confidence threshold. The YOLO algorithm will give a confidence calculation, from 0 to 1 determining how likely it is that the object is actually what it says it is. We decided that we want a 85% threshold. Aichen and I met up and we kept looking into the file structure of the YOLO code, and found that the given YOLO_imgs text files in the dataset are actually the labels that the train.py code is looking for. We will have to modify the script that segmented the dataset images into train/val/test and also segment the corresponding label text files. We have ordered all the materials and some of them have started arriving. We will definitely start building next week as our materials start arriving. We will also be spending the week working on the design document. I think we are still behind on the ML side, as our file structure is still having issues. But there has been development in terms of the detection portion of the software. 

Posted on

Ting’s Status Report for 2/18

We continued debugging the object detection code, adding code to clean up and move around the file system. The code is able to run halfway through, but it is still getting stuck on the error of not having the labels file. Our problem is that the file structure is not set up right for the dataset that we’re using with the code that we are modifying. I believe we are behind in this aspect, that we should get the code running.

I also worked on the mechanical design, refining the placement of the neopixel, and also changing it to a strip of neopixels. We thought about making a backboard to support the platform that the camera and jetson to sit on, but we ended up keeping the PVC pipe idea in order to hide the wiring. 

We all worked together on the design presentation, which I will be presenting this week, making block diagrams as well as FSMs and flowcharts to represent the flow of the data throughout the system.

We are a bit behind in terms of mechanical building since we have not finalized the parts to order. But we are finalizing and submitting the order form tonight.

Classes used to develop our design: 18240, 10301 Intro to ML, 18100 Intro to ECE

Posted on

Ting’s Status Report for 2/11

Our team worked together to lay out the use case requirements, scope and schedule of our project for the project proposal presentation. We chatted after the presentation based on some of the questions brought up by other students and the TA. One example was if a student could put a whole tray of objects onto the bin platform. We decided that if we wanted to consider this case in the future, we would have to use detection as well as classification. So we decided to stick with YOLO instead of just Resnet, since resnet would only have the classification part.

This week I did research on the appropriate machine learning model that can be used to perform classification of a plastic bottle. I found a We searched for datasets, and found one that can perform classification of 4 types of drinkable waste. With this dataset I believe we can expand our MVP from only classifying plastic water bottles to also being able to classify milk cartons, glass, and aluminum cans. I started to use some starter code for YOLO to run the code with this dataset I found, and am in the middle of debugging to get the code to run all the way through. For the trapdoor I researched different videos of mechanisms: one would be a servo with an arm that is pulling and pushing the door. The other would be a ledge that retracts to let the door fall open, and an actuator to push the platform back up. Next week hopefully we can start training and also research more specifics of the mechanical aspect of the project.

Drinking waste dataset: https://www.kaggle.com/datasets/arkadiyhacks/drinking-waste-classification