Category: Daniel’s Status Reports

We as a team met earlier this week to hash out our integration plan. Opalina was to send me a revised YOLO trained model which has updated signage recognition, and I was to integrate that model in my already-working LLM and OCR script. For this week, I finetuned and prepared my scripts so that when the models are finished training the integration will be as smooth as possible.

We met today to start the integration process for the demo next week. I have already finished OCR and the UI, and now we are trying to combine them with the YOLO. I have coded the script with Opalina where the system takes the inputs from the OCR and YOLO, and translates it into directional guidance.

This was a busy week. Throughout the days, I worked on the OCR aspect of the project. So far, I’ve been working on a script that allows for a repeat of words or instructions on the sign in a photograph. The output is a print statement so far, and the next thing to do is to work on the CV aspect of the project. Once both are done, I can integrate my LLM models and actually have the program repeat instructions.

Over the week, I focused on getting OpenCV to work. Now that I’m getting sent the YOLO script for sign detection, I’ll integrate it into what I got, and produce an output that the LLM models can translate. Hopefully, this gets done over the next few days, Tuesday at the latest.

Spent some time focused on finishing up the tweaks on the LLMs that I mentioned that I was working on last status report. Also started to work with OpenCV to help with the object detection part of the project, since that requires a hefty workload which will need multiple people to get working. So far getting OpenCV to work has been difficult but I should make some solid headway through the next few days.

As well as preparing for the slides and presentation for the Design Report, I have been working on the audio feedback part of the system. Audio integration setup has been done, and I’ve been working on implementing VOSK and Google TTS, and testing it myself. I’m in the process of creating a simple script that just repeats whatever I say back to me, which I’ll be able to present to my group on our meeting on Monday.

After dividing the work, it was agreed that I would work on the quantitative requirements, testing, verification, validation, and implementation plan. So far, most of my time was spent on the implementation plan. I created a list of items that we would need to buy for our project, as well as detailing the plan on how to implement the project on the software side. Currently leaning towards using YOLO, Google TTS, VOSK, and Python Threading.

After receiving feedback for our project during the presentation, we’ve agreed to meet next week to discuss what we have researched and discovered. As promised in the slides, I started to review existing AI object detection models. So far YOLO v8 model seems to impress. I’ve learned that at least 10k images would be needed, so this is something to keep in mind moving forward.