This week I spent a lot of time working on the translation on the iOS side. I was able to utilize VNRecognizeTextRequest in Apple’s Vision framework to who real time live translation from and to multiple languages. Furthermore I was able to show the translations and their respective bounding boxes over the live video for debugging purposes and to verify that it was working properly.
I had to do some additional research via google searches and Apple’s documentation to get this working to the desired performance level. One helpful parameter I discovered was the VNRecognizeTextRequest recognitionLevel. Changing from “fast” recognition to “accurate” significantly improved character recognition performance and hardly if at all increased latency of the model.
Overall, I’m really happy with the state of the translation, although there is still some work on handling text groupings and a large quantity of text. I think we’re definitely still on schedule to complete and survey the Dawggles this week.