Author: okhanna

The new camera arrived this week, which meant we had to start the process of hardware integration from scratch. The new AI hat, while easy to set up, is posing compilation issues with the custom ML models. While the text-to-speech and sign detection seems to be acceptably, this week, we hope to make more progress on integrating the camera with the software. Once we have a basic level of functionality, we hope to start user and battery tests to ensure that the device holds up to the initially outlined use-case requirements (accuracy, latency, power, depth perception etc)

This week, I used new training and testing datasets to fine-tune the YOLO model that we previously used for our interim demo. The new model looks for an added set of features in order to reduce the amount of OCR passes needed on each image, subsequently reducing our latency. I ran a few standard tests in order to verify the functionality of this new model, and found a slight drop in accuracy, which I am currently working to reduce. In the coming week, I plan to build new custom test datasets in order to properly identify these points of error.

This week, I managed to run and fully test YOLO and OpenCV in conjunction with OCR, to extract and interpret all the relevant information from an airport sign. This script simply needs to be run with the TTS component in order to complete the end-to-end software subsystem. In the coming weeks, we hope to use camera footage to run the models on the Pi.

Currently, the individual components seem to be functioning adequately, including the OCR, YOLOv8 model, and the camera. However, the camera is working slower than initially anticipated, posing a potential risk of the model’s not running properly during integration. In the upcoming week, we aim to test this hypothesis and make changes to our models or equipment as necessary.

This week, I managed to fine-tune the YOLO model while figuring out the semantics of the OpenCV, YOLOv8 and OCR integration. During the coming week, I hope to get the ML system fully functioning and at least partially integrated, ready to run on the Pi.

The YOLO model is fully trained and functional on large airport datasets (with a variety of images). OCR is proving a little more difficult to integrate into thew software subsystem, but I hope to have that figured out by the end of this week. The only potential issue we see right now, is the model not running fast enough on the Pi, as local tests prove that it might be slower than anticipated.

This week I dived deeper into finalizing and testing YOLO v8 for the sake of object detection. I researched ways to fine tune to model to fit our purposes while eliminating potential for error due to crowds, shaky cameras, advertisements, etc.