Author: rsbaschn

This week I worked on deploying the ML models to the raspberry pi. This consisted of setting up the python environment, converting the Jupyter notebook into a standard python file, and setting up the file structure the raspberry pi will use. Since the notebook displays the bounding boxes and images when inferencing, when converting to a python file, I removed this code for faster performance since the end user will not see this anyway. I tested this implementation with a sample image that had two license plates in plain view. This is the same image used when testing the Jupyter notebook in Google colab. The program ran in just over 23 seconds, which should be plenty fast enough for our 40 second timing requirement. The models I used were the NCNN models but no quantization was used, so this number can be easily lowered further if needed. The code can be found here. When setting up the file system, I put the pictures and models into their own folders to easily switch between models and the images I test. Last week I worked on the design report, where I focused on the system implementation as well as the design trade studies.

My progress is on schedule. By next week I hope to finalize the MVP of the dashcam side of things, and shift focus to setting up the cloud

This week I worked on the presentation with Tzen-Chuen and Eric, especially with regard to the details of the implementation, such as using YOLOv11 and AWS Lambda in our final design. In addition, I worked with Eric on setting up the pipeline of YOLOv11 to PaddleOCR. I made two versions of the pipeline, one that first detects cars and crops those images, then into license plate detection and cropping, then finally PaddleOCR to read the license plate. The second one does not do the initial car cropping and goes straight into license plate detection. The google colabs can be found here and here. I also did some more research on how to deploy the models to the raspberry pi, and found that we should use NCNN models. For our mvp, we will use a python script that I am working on running on a headless os using the optimized models. As soon as the camera arrives, we should be able to make a basic MVP excluding the cloud server.

My progress is on schedule. By next week, we hope to have a dash cam module MVP and get metrics on the initial performance of the device.

This week I worked more on the YOLOv11 model as a possible model for license plate detection. I set up a workflow to make training these models very easy on the ECE machines so that we do not waste our budget. After doing this, I trained the YOLOv11n model for both 100 and 400 epochs to get a good baseline model for testing on our raspberry pi and getting to MVP as quickly as possible. The models can be found here. The precision of the 400 epoch model is 0.984 and the recall is 0.964 on the validation dataset. After training, I researched the best model file types to which I should export my trained model, and I discovered that TensorFlow Lite is a good option, especially for edge devices. After some technical problems, I was able to export the Pytorch-trained model to ONNX and then to TFLite. I tried to export with INT8 quantization to further improve performance on an edge device, however, I faced many difficulties and was not able to do it.

My progress is on schedule. Next week I hope to load the model onto the Raspberry Pi and test its performance metrics on the device, specifically on how long inference takes, so that I can figure out if I want to use a different size model or if I need to get quantization working.

This week I worked on the presentation slides, researching amber alert data/statistics, competitors such as Plate Recognizer, and the requirements for our project, as well as adding detail and polish throughout the presentation. After some research into how others have implemented license plate detection, I worked on investigating the feasibility of using YOLOv11 as a base model for license plate detection, since it is popular in the space, and have looked into ways to fine-tune the model for this use case. I have made a Jupyter Notebook file that will tune the model for license plate detection using an open-source dataset I found online, and I have explored ways to have this training run on sites like Google Colab and Kaggle, as well as on the CMU ece machines. Right now I am learning toward using the ece machines due to the strict GPU restrictions with using google colab and kaggle for free. The link to the jupyter notebook is here.

My progress is on schedule. Next week I hope to have completed training on the YOLOv11 nano model on the dataset for around 100 epochs and get some preliminary data on its performance. I would also like to find out how to load the model on a Raspberry Pi and get some metrics on how fast this trained model will run on the device.