Team A3: Where is the Barcode?

This week, Jiyeon was mainly worked on preparing final presentation and web application. She has presented the final presentation on Wednesday so she spent some time making slides and practicing the presentation. After that, she started working on web app and integrating her model with it. She’s using Django web framework for her web app. Current progress on web app is real-time streaming and capture and save an image from webcam. She’s working on integrating her YOLOv5 model with the web app and creating item display page. After the demo, she will be working on final video and report.

Raymond

So last week, right, I was primarily focused on finding a way to remotely control the conveyor belt, and to my surprise, it worked. Thanks for the help from the experienced professor and my TA, without expert knowledge, we would waste a lot of resources in testing different design method, and probably we wouldn’t be able to finish the design on time.

Surprisingly this is the first hardware engineering project I made from scratch. Some of the important things I learned from last week, and the semester, was that in the future, if I want to do certain project. I need to

1. Spend more time design than implement.
2.  Network and find the people who specializes in this field(sometimes an expert idea is worth millions of dollars, if money is the easiest understand metrics of impact).
3.  Always document what I did, and write a journal of the engineering process.
4. Always try to work in person with teammates. Work from home works for solo projects, but it never works for a team, no matter how easy the project seems to be.

This week, Jiyeon mainly worked on preparing dataset and training the YOLOv5 model for image classification. Since Rachana has dropped the course last week, she took over her part and started to work on it. She explored some options like YOLO and MobileNet, and decided to train YOLOv5. She took a total of 400 images of Advil and Tylenol in different angles, and annotated them with Roboflow. She split the dataset into three, which are train (70%), valid (20%), and test (10%), and trained the YOLOv5. So far with two classes (Advil and Tylenol) and 40 test images , her model has a 100% accuracy. She’s hoping to get more dataset and train her model to classify more objects.

Rescope plan for Jiyeon: Initially, we were aiming for 10 different objects, but now Jiyeon is targeting 5 different objects. Also, the web app will be very simpler than the original plan. For now, Jiyeon’s goal is to have the item and price displayed. If time allows, Jiyeon is hoping to add more features.

So last week, right, I was primarily working on setting up the raspberry pi and preparing for the final report.

While I was writing the final report, I realized that our design really has changed a lot since the beginning. Primarily because when I designed the hardware, I barely had any handson experience with any hardware in my life. I grew up in China which follows the Soviet education system, following mostly on the theoretical aspect in STEM rather than the hands on part.

We are not behind the schedule.

For next week, I plan to discuss with my professor and TA on whether to drill other holes or working on the micro-controller part.

Rescope plan for Mingrui Yang:

Instead of having a long conveyor belt, we plan to only have it as the length we have now. We do not even have a working shorter version, so making it longer seems risky because that might destroy the wood, and ending up with nothing working at all. Based on my previous experience, it is always safe to start small rather than being ambitious on our first complete engineering project.

Jiyeon:

This week, Jiyeon finished text orientation and improved efficiency and readability of my code. She initially got error from Tesseract’s image_to_osd function, which corrects the text orientation, but she couldn’t fix the error. So instead of using image_to_osd function, she wrote a function to check the skewness and correct the image. Additionally, her other main concern was efficiency. This week, she worked on improving efficiency and readability of her code by breaking down my code into multiple helper function and files and making sure the code stops if a matching text is found from our product info dictionary. She also wrote a helper function to save our product info to json file which would be useful for web app later. She made a good amount progress on the efficiency but this definitely still needs some improvement since the user would not want to wait ~6 sec each product to be scanned. For next week, she is hoping to improve more efficiency with consistent test images.

Raymond:

So for last week right, Raymond was primarily working on a  couple of design challenge. What he have observed is that there are similarities in hardware and software engineering. The similarity is that there are challenges along the way to finish a project, and sometimes when we see a challenge he has to find ways to pass the challenge. In software, it might be switching to different APIs, different library vendors etc. Sometimes one function call provided by one party works, and the others don’t, and often we don’t know why(probably we didn’t understand the input well enough, the output well enough, and the overall function well enough. A lot the times this happens in math as well, when we don’t know the exact size and dimension and type of a variable, and we don’t know the meaning of certain symbol, the abstract and geometric meaning, and we don’t understand it well enough it often leads us to misuse it, forgetting to use it, not understanding the derivations, etc.

So what he learned last week is that in hardware, we also need to have some basic understanding in the laws of physics and the properties of the materials. He thinks last week most of the knowledge domain he utitlized was from Physics I, Newtonian physics, in terms of friction, surface area, torque. Some basics application of physics(not numerial, but some high level concepts).

Anyways, he get the belt working at half of the length. Now the problem is our design is not able to carry a lot of the weight because we have hollow areas on the belt, but since this is a proof of concept we will just put light weight stuffs on. For next week, he plans to focus on raspberry pi. We are on schedule.

Overall, Raymond and Jiyeon made great progress this week. We will tackle problems together next week using our knowledge base.