Posted on

Vicky’s Status Report for 4/12

Personal Accomplishments

    • Dash Cam Bringup:
      • Added a dip switch and software script to allow users to opt in or opt out of the ALPR mode
      • Refactored GPS code to make it nonblocking when GPS signal is not detected and debugged GPS connection issues on power on
      • Implemented sync match image chunk upload and experimented with async match upload using thread
        • Both HTTP GET and HTTP POST use the same I2C address, so I have to implement a locking mechanism
        • This results in a traffic jam that we think is not worth for implementing async chunk image upload
        • So we decided to stop further inferencing when we are uploading a match image
    • Dash Cam Testing:
      • Added an LED to indicate the stage of the dash cam for easier road testing, LED is ON during setup, LED flashes slowly during recording, and LED flashes faster during inference
      • Wrote a pure recording script to record real-road footage for future demos
      • Worked with Christine and Andy to conduct a 30 minute road test under rainy setting
    • Dash Cam Verification:
      • Verified that the dash cam records continuously in 1-minute clips for 10 minutes in a stationary setup
      • Verified that the dash cam records continuous footage under five separate real-driving tests of at least five minutes each
    • ALPR Verification:
      • Benchmarked the YOLOv11n model using a curated set of 386 real-world images sourced from the Kaggle-based dataset, achieving 90.4% mAP50
      • Benchmarked the PaddleOCRv3rec component on the scraped dataset from platesmania.com, which includes 4,000 cropped synthetic U.S. license plate images and 1,000 cropped real-world images, achieving 93% accuracy
      • Benchmarked the end-to-end ALPR pipeline by passing 5,000 full-frame, real-world images from platesmania.com through the system, achieving 81% accuracy

Progress

My progress is on schedule.

Schedule

  • Road tests
  • Dash cam pipeline fixes (API key integration, etc.)
  • Scripts for demo purposes
Posted on

Vicky’s Status Report for 3/29

Personal Accomplishments

  • CI/CD:
  • Dash Cam Bringup:
    • Integrated HTTP GET code to end to end script and got cache working to reduce the number of requests
    • Integrated Andy’s UPS auto shutdown code to end to end script
    • Integrated Sparkfun GPS-RTK module via USB serial to acquire GPS coordinates in the end to end script
      • I2C connection was unsuccessful due to address conflict with the UPS module
      • UART connection was unsuccessful for unknown reasons
    • Added logging to end to end script, with auto ALPR on when power on and board shutdown when unplugging the cable
    • Improved end to end script to only run OCR on horizontal detection boxes with >0.5 confidence and only send GET requests when recognized plate length is between 2 and 8 to reduce the number of requests
    • Assisting Blues engineer to debug the GET json error
  • Dash Cam Testing:
    • Set up the dash cam to be installable on a car with a phone holder
    • Worked with Christine and Andy to road test the dash cam in real world setting and validated ALPR and GET request capabilities

Progress

My progress is on schedule.

Schedule

  • Central server integration and testing
  • Opt-in switch module bringup and testing
  • End-to-end pipeline debugging
Posted on

Team Status Report for 3/22

Risks

  • Communicating between the dash cam and the central server is a challenge. We are trying to send a web get request from Notehub to our AWS public http endpoint. We are able to get expected json response using manual curl as well as copying the link into web browser. However, when we set up the route in Notehub and get it this way, we always receive {‘result’: 503, ‘body’: {‘err’: ‘hub request error: response from web service does not appear to be JSON {not-json}’}}, and central server also doesn’t receive any GET requests. We tried setting up another public api route, which we are able to web get with Notehub, indicating that the route setting should work. We are adding more logging on the central server side, posted the issue on Blues forum, and contacting other teams also using Blues.

Changes

  • Blues Notecard doesn’t support running cellular and GPS at the same time. If one is in continuous mode, the other one needs to be in periodic mode with at least 5 minute interval. Since 5 minute interval is rather long, we are planning to purchase an additional GPS module. In the mean time, we will use the GPS location obtained by cellular data for geolocation tagging.

Schedule

Posted on

Vicky’s Status Report for 3/22

Personal Accomplishments

  • Dash Cam Bringup & Testing:
    • Worked with Andy to connect Blues Notecard to Notehub
    • Worked with Christine to send HTTP GET requests to central server
      • Debugging why parsing the server response as json fails, current workaround is to take it as plain/text then decode it
    • Finalized Adafruit Ultimate GPS as the separate GPS module
    • Wrote launcher script to automatically run camera recording and detection at power on
    • Debugging the end-to-end pipeline

Progress

My progress is on schedule.

Schedule

  • Order GPS module
  • Central server integration and testing
  • Opt-in switch module bringup and testing
  • End-to-end pipeline debugging
Posted on

Team Status Report for 3/15

Risks

  • Communicating between the dash cam and the central server will be challenging, as we need to determine how they communicate and how the dash cam locates and uploads images. Since we’re no longer building a web app, Andy can focus more on integration, and we plan to implement this in blocks to manage complexity and reduce risks.
  • Determining how images and metadata are stored on the dash cam is an issue we talked about and need more discussion on tradeoffs. We discussed storing the information about the license plate number that we sent on dash cams so we do not send them again in recent times. The benefit of this would be to reduce flushing to end users, but this introduces more computational complexity on Raspberry Pi and might cause unconfident images only being sent once that are not enough for human verification. We also need more discussion on if we should store temporary images on RAM and send them to the server directly, or we should always store them to SD card first and send from SD card.

Changes

  • Based on feedback from our design report, we decided to switch to an API-first approach instead of a full web app, as it better aligns with our system’s intended use. Since law enforcement agencies likely already have their own software, exposing an API enables seamless integration rather than requiring them to adopt a separate interface. Moving forward, we plan to expose a single POST endpoint for adding plates to the watchlist. To ensure security, external services will be required to use an API key to access the endpoint, and we will implement audit logging to track who added what. If a match is detected, we will log the event as if notifying a tip line—without actually sending notifications. This keeps the system lightweight and secure while still preserving important tracking data.
  • Based on dash cam bringup experimentations, we found writing raw footage to SD card to be the performance bottleneck, limiting the frames per second the dash cam can capture. To address this issue, we decided to store raw footage at 480p, while still having the dash cam inference on higher resolution (2304*1296) frames.

Schedule

Posted on

Vicky’s Status Report for 3/15

Personal Accomplishments

  • Dash Cam Bringup & Testing:
    • Setted up Python virtual environment to run detection and OCR models with streaming Camera Module 3 input
    • Deployed and benchmarked ML license plate detection model on RPi 5, PyTorch format achieves around 370ms latency per frame, ONNX format achieves around 350ms latency per frame, and NCNN format achieves around 160ms latency per frame with streaming camera input
    • Deployed and benchmarked ML license plate OCR model on RPi 5, Paddle format achieves around 120ms latency per frame with streaming camera input, and ONNX format was deployed unsuccessfully
    • Wrote the end-to-end pipeline, with the main thread recording footage in 1 minute 480p20 clips, and the inference thread inferencing 2304*1296 resolution images and storing cropped license plate results at around 3fps

Progress

My progress is on schedule.

Schedule

  • GPS module bringup and testing
  • Network module bringup and testing
  • Opt-in switch module bringup and testing
  • End-to-end pipeline debugging
Posted on

Vicky’s Status Report for 3/1

Personal Accomplishments

  • Design Report:
    • Wrote and edited the design report
  • ML License Plate OCR:
    • Cleaned up platesmania.com dataset through script and manual inspection to improve training quality
    • Benchmarked a variety of OCR models and selected en_PP-OCRv3_rec model for its ease of integration with Python and lesser likelihood to overfit (93% accuracy on platesmania.com 80% synthetic + 20% real-world license plate dataset)
  • ML End-To-End:
    • Designed, implemented, and tested the end-to-end script, achieving 79% end-to-end accuracy with platesmania.com 100% real-world dataset, with 17% false positive, and 4% false negative
  • Dash Cam Bringup & Testing:
    • Collaborated with Andy to bringup and test the RPi 5 board and camera module 3

Progress

My progress is on schedule.

Schedule

  • Single-board computer bringup and testing
  • Camera module bringup and testing
  • GPS module bringup and testing
  • Network module bringup and testing
Posted on

Team Status Report for 2/22

Risks

  • A risk we anticipate is that the power source circuit or the UPS hat does not work as we desired. For example, the voltage dropped too quickly that is not long enough for the rpi to execute its shut down process, or the circuit overheats because of the capacitor. To mitigate this risk, we could
    • Add more element in our circuit, for example a cooling device
    • Add a button connected to the rpi for the user to manually control the rpi to execute its shut down process

Changes

  • We originally planned to use DynamoDB for the web app backend since it’s easier to set up and works well for flexible data storage. But after thinking more about security, especially tracking access logs, we decided that a hybrid approach might be a better fit. We want to use RDS for better logging and auditing to improve security and accountability, while DynamoDB will handle personal watchlists for fast lookups. This balances performance with the need for structured access tracking. (updated web app database schema)
  • During last week’s status report, we identified the need for a circuit to support the dash cam because the car’s power supply shuts off immediately when the engine is turned off, while our RPi5 requires additional time to properly shut down to prevent data corruption or potential damage. After designing a preliminary solution, we determined that incorporating a supercapacitor would be effective, as the shutdown process takes approximately five seconds. While researching, we also came across a pre-made module called the UPS HAT for the RPi, which addresses a similar issue. However, we would like to discuss in next week’s meeting whether our custom-designed circuit or the UPS HAT would be the better fit for our overall design. (UPS hat)

Schedule

Posted on

Vicky’s Status Report for 2/22

Personal Accomplishments

Progress

My progress is on schedule.

Schedule

  • Single-board computer bringup and testing
  • Camera module bringup and testing
  • ML models deployment and benchmarking
  • Design report
Posted on

Vicky’s Status Report fot 2/15

Personal Accomplishments

  • ML End-To-End Pipeline Design:
    • Updated ML flow: Frame Acquisition -> License Plate Detection -> License Plate OCR -> Text Formatting -> Transfer OCR Detected Text to Handshake with Cloud -> Transfer Cropped License Plate Frame with Time and GPS Info to Cloud if Applicable
  • ML License Plate Detection:
    • Experimented with frame preprocessing using CV, results undesirable
    • Finetuned a Yolov11n model on Kaggle dataset, achieving 90.4% mAP50 and 65.5% mAP50-95
  • ML License Plate OCR:
  • Dash Cam Design
    • Finished dash cam v1.0 block diagram design
  • Hardware Purchase
    • Raspberry Pi 5 as the SBC
    • Raspberry Pi Camera Module 3 + cable
    • Blues Notecarrier Pi + Notecard Cellular + GPS&Cellular antenna

Progress

My progress is on schedule.

Schedule

  • RPi 5 bringup and testing
  • Camera module bringup and testing
  • License plate detection model deployment and benchmarking
  • License plate OCR model training, deployment, and benchmarking
  • Design report