We met today to start the integration process for the demo next week. I have already finished OCR and the UI, and now we are trying to combine them with the YOLO. I have coded the script with Opalina where the system takes the inputs from the OCR and YOLO, and translates it into directional guidance.
Opalina’s Status Report 3/29
This week, I managed to run and fully test YOLO and OpenCV in conjunction with OCR, to extract and interpret all the relevant information from an airport sign. This script simply needs to be run with the TTS component in order to complete the end-to-end software subsystem. In the coming weeks, we hope to use camera footage to run the models on the Pi.
Krrish’s Status Report 3/29
I’m running experiments with the camera and YOLO model. It is struggling and working quite slowly with significant delays. I’ve put together a list of new camera options and have narrowed it down to pretty much 2 options. I will place an order on the one i think will work best.
Team Status Report 3/29
We have OCR and the YOLO model working independently. We are working on integrating that and also testing on videos rather than still frames. Our goal is to have this working on our laptops and then integrate it with the Pi. Simultaneously we are testing the camera on the Pi with the YOLO model to see how fast it would run. Currently it looks like the camera is running too slow on the pi and so we’re looking at different options. We’ve narrowed it down to 3 options and will place an order today.
Daniel’s Status Report 3/22
This was a busy week. Throughout the days, I worked on the OCR aspect of the project. So far, I’ve been working on a script that allows for a repeat of words or instructions on the sign in a photograph. The output is a print statement so far, and the next thing to do is to work on the CV aspect of the project. Once both are done, I can integrate my LLM models and actually have the program repeat instructions.
Krrish’s Status Report 3/22
I’m doing research into new camera that we can use since the old camera is not compatible with the AI accelerator. I’ve also made progress on the CAD model.
Team Status Report 3/22
Currently, the individual components seem to be functioning adequately, including the OCR, YOLOv8 model, and the camera. However, the camera is working slower than initially anticipated, posing a potential risk of the model’s not running properly during integration. In the upcoming week, we aim to test this hypothesis and make changes to our models or equipment as necessary.
Opalina’s Status Report 3/22
This week, I managed to fine-tune the YOLO model while figuring out the semantics of the OpenCV, YOLOv8 and OCR integration. During the coming week, I hope to get the ML system fully functioning and at least partially integrated, ready to run on the Pi.
Daniel’s Status Report 3/15
Over the week, I focused on getting OpenCV to work. Now that I’m getting sent the YOLO script for sign detection, I’ll integrate it into what I got, and produce an output that the LLM models can translate. Hopefully, this gets done over the next few days, Tuesday at the latest.
Team Status Report 3/15
Our most significant risk is the RPi not being sufficient enough to handle the camera input, object detection processing, and handling the output. The risk is that there will be latency issues, and the contingency plan as of now is to switch to offline LLM models to reduce the load on the RPi. Currently there have been no changes to the existing design of the system, and no changes to the schedule, as of right now we are on track.