Posted on

Richard’s Status Report for 3/15

This week I focused on the integration between the Raspberry Pi and the cloud.  I worked on creating the database update code, where a Python function will retrieve the latest version of the Amber Alert database from the supabase and update the locally saved database with it. If the Raspberry Pi does not have an internet connection, it will simply continue using the locally saved version.  I also wrote the code to check for matches in the database. A loop checks the OCRed text with the database and if a match is found, the original image is uploaded to a supabase bucket, and an entry is made into a “possible matches” table for further verification by the cloud models. We will later integrate Eric’s edge function to move more processing to the cloud. The updated code can be found here.

My progress is on schedule. By next week, I hope to finalize the locally run code and work on the edge function that should run when a match is uploaded to corroborate with larger models.

Posted on

Team’s status report for 3/8/25

The most significant risks have remained unchanged, with it being the delay of MVP. These risks are being mitigated by having spare components ordered from ECE inventory to substitute instead of our actual desired components. In terms of contingency plans, there are none, as MVP is a crucial step that cannot be circumvented in any way. However, we are testing the parts of our design as they finish, so we are confident that they will work as we assemble them into an MVP.

We made a few changes to the system design. We updated the cloud reliability target from 95% to 97% to reduce downtime risks and ensure timely database lookups for license plate matching, as AWS’s baseline uptime guarantee is closer to 95%. This is pretty realistic and also follows the published statistics for server uptime on AWS and shouldn’t change our costs. We also refined the edge-to-cloud processing pipeline to improve accuracy and efficiency. Both high and low-confidence detections are sent to the cloud, but low-confidence results also include an image for additional verification using more complex models. This change ensures that uncertain detections receive extra processing while still keeping the system responsive and scalable.

This will not significantly alter the current schedule, as lowering the accuracy requirement will make training easier and potentially quicker.

In addition, we have written the code for using the ML models on the raspberry pi and it can be found here.

Part A (Richard):

Our design should make the world a safer place with regards to child kidnapping. Our device, if deployed at scale, will be able to locate the cars of suspected kidnappers using other cars on the road quickly and effectively, allowing law enforcement to act as fast as possible. While we currently only plan on using the device with amber alerts, a US system, the design should largely work in other countries. The car and license plate detection models are not trained on the cars and plates of any specific country, and PaddleOCR supports multiple languages (over 80) if needed with foreign plates. This means that if other countries have a similar system to amber alerts, they can use our design as well. Our device may also motivate other countries who do not have a similar system to start their own in order to use our design and better find suspected kidnappers in their country.

Part B (Tzen-Chuen):

CALL sits at a conflicting cross-section of cultural values. Generally, our device seeks to protect children, a universal human priority. It accomplishes this through a distributed surveillance network, akin to the saying “it takes a village to raise a child.” By enabling a safer, more vigilant nation, we are in consideration of a global culture.

In terms of traditional “American values,” CALL presents a privacy problem. While not explicitly a constitutional right, it is implied in the 4th amendment. A widespread surveillance network is bound to raise concerns among the general public. We attempt to mitigate this concern by only sending license plate matches that have a certain confidence level to the cloud, and never to the end users. This way we balance the shared cultural understanding of child protection with the American tradition of privacy.

Part C (Eric):

Our solution minimizes environmental impact by leveraging edge computing, which reduces reliance on energy-intensive cloud processing, lowering power consumption and data transmission demands depending on the confidence of the edge model output. The system runs on a vehicle’s 12V power source, eliminating the need for extra batteries and reducing electronic waste. Additionally, its modular design makes it easy to repair and update, extending its lifespan compared to full replacements. These considerations ensure efficient operation while reducing the system’s environmental footprint.

Posted on

Richard’s status report for 3/8/25

This week I worked on deploying the ML models to the raspberry pi. This consisted of setting up the python environment, converting the Jupyter notebook into a standard python file, and setting up the file structure the raspberry pi will use. Since the notebook displays the bounding boxes and images when inferencing, when converting to a python file, I removed this code for faster performance since the end user will not see this anyway. I tested this implementation with a sample image that had two license plates in plain view. This is the same image used when testing the Jupyter notebook in Google colab. The program ran in just over 23 seconds, which should be plenty fast enough for our 40 second timing requirement. The models I used were the NCNN models but no quantization was used, so this number can be easily lowered further if needed. The code can be found here. When setting up the file system, I put the pictures and models into their own folders to easily switch between models and the images I test. Last week I worked on the design report, where I focused on the system implementation as well as the design trade studies.

My progress is on schedule. By next week I hope to finalize the MVP of the dashcam side of things, and shift focus to setting up the cloud

Posted on

Team’s Status Report for 2/22/25

Currently the most important risks that could jeopardize the success of our project is the MVP being delayed by any reason, as getting the MVP off the ground and tested will reveal any weak points that we need to address. The MVP being delayed will likely mean we will be time crunched when trying to iterate. 

We made a modification to the timing requirements based on further research into the Amber Alert use case after receiving feedback. Initially, the system was designed with a 60-second processing requirement, which aligns with the average lane change frequency on highways (2.71 miles). However, after analyzing worst-case merging scenarios, which would require about 20 seconds, we found 40 seconds would be a more appropriate constraint for the MVP to shoot for as a middle ground between these two cases, which once achieved, we would continue to target that worst-case timing requirement. This would better ensure timely license plate detection before a vehicle potentially exits the field of view. This wouldn’t have any direct costs, but it may affect the requirements we have on the processor, depending on how long it takes to do model inferencing.

Another change we are making is moving to supabase for our backend server, as it presents a much more user-friendly interface for our use case targets (law enforcement, amber alert) and is more setup-friendly.

Our schedule has not changed.

In addition, we have worked on our camera to OCR pipeline, and have made two versions of the code we will use: version 1, version 2.

Posted on

Richard’s Status Report for 2/22/25

This week I worked on the presentation with Tzen-Chuen and Eric, especially with regard to the details of the implementation, such as using YOLOv11 and AWS Lambda in our final design. In addition, I worked with Eric on setting up the pipeline of YOLOv11 to PaddleOCR. I made two versions of the pipeline, one that first detects cars and crops those images, then into license plate detection and cropping, then finally PaddleOCR to read the license plate. The second one does not do the initial car cropping and goes straight into license plate detection. The google colabs can be found here and here. I also did some more research on how to deploy the models to the raspberry pi, and found that we should use NCNN models. For our mvp, we will use a python script that I am working on running on a headless os using the optimized models. As soon as the camera arrives, we should be able to make a basic MVP excluding the cloud server.

My progress is on schedule. By next week, we hope to have a dash cam module MVP and get metrics on the initial performance of the device.

Posted on

Richard’s Status Report for 2/15/25

This week I worked more on the YOLOv11 model as a possible model for license plate detection. I set up a workflow to make training these models very easy on the ECE machines so that we do not waste our budget. After doing this, I trained the YOLOv11n model for both 100 and 400 epochs to get a good baseline model for testing on our raspberry pi and getting to MVP as quickly as possible. The models can be found here. The precision of the 400 epoch model is 0.984 and the recall is 0.964 on the validation dataset. After training, I researched the best model file types to which I should export my trained model, and I discovered that TensorFlow Lite is a good option, especially for edge devices. After some technical problems, I was able to export the Pytorch-trained model to ONNX and then to TFLite. I tried to export with INT8 quantization to further improve performance on an edge device, however, I faced many difficulties and was not able to do it.

My progress is on schedule. Next week I hope to load the model onto the Raspberry Pi and test its performance metrics on the device, specifically on how long inference takes, so that I can figure out if I want to use a different size model or if I need to get quantization working.

Posted on

Team’s Status Report for 2/8/25

The most significant risk is likely not being able to get the edge-compute model working well enough in time, and not having enough time to switch our integration strategy to have the license plate recognition happen on the cloud. As such, we are looking into both edge-compute models as well as models that we could use to run on the cloud, and are considering how we would integrate them in each scenario so that any necessary transitions can be made without too much trouble.

The design was not solidified before this week, but the fundamental requirements have been selected, namely image recognition latency and plate detection range. These “changes” are necessary as we need concrete and realistic goals to work towards while building our design. The costs that this change incurs are minimal, as the design was not formalized previously. 

Since nothing has changed from our plans, only that our design approach is solidifying, we have not made any changes to the schedule. However, we are looking into how we can make an MVP as early as possible to begin testing early so any major changes that need to be made will happen earlier in the process.

In investigating models for license plate detection we have made a jupyter notebook for training YOLOv11 for license plate detection, linked here.

Posted on

Richard’s Status Report for 2/8/25

This week I worked on the presentation slides, researching amber alert data/statistics, competitors such as Plate Recognizer, and the requirements for our project, as well as adding detail and polish throughout the presentation. After some research into how others have implemented license plate detection, I worked on investigating the feasibility of using YOLOv11 as a base model for license plate detection, since it is popular in the space, and have looked into ways to fine-tune the model for this use case. I have made a Jupyter Notebook file that will tune the model for license plate detection using an open-source dataset I found online, and I have explored ways to have this training run on sites like Google Colab and Kaggle, as well as on the CMU ece machines. Right now I am learning toward using the ece machines due to the strict GPU restrictions with using google colab and kaggle for free. The link to the jupyter notebook is here.

My progress is on schedule. Next week I hope to have completed training on the YOLOv11 nano model on the dataset for around 100 epochs and get some preliminary data on its performance. I would also like to find out how to load the model on a Raspberry Pi and get some metrics on how fast this trained model will run on the device.