Blind and visually impaired people often face significant barriers when attempting to organize and identify objects in their day to day life. The most troublesome can be similarly shaped items with no tactile indicators like medication bottles, soup cans, and cardboard food packaging. This raises significant safety, efficiency, and independence concerns. Current braille labelling options do exist, but are limited. There are present problems including the devices being bulky, stationary, and expensive, in addition to sometimes requiring the assistance of a sighted user.
We have created a portable, battery powered device that can identify English writing on unknown labels and automatically print accurate braille labels. To emphasize independence, after an item image is captured, the text that is pulled from the image is announced on a speaker. The user then has the ability to continue or cancel the print. This allows the user to scan multiple sides of an item since they aren’t able to see which side has the label. Our product fell short in the force created by the pins, so paper instead of plastic labels were used for demonstration, but braille was successfully printed on the paper. This force issue is a next step for our device. We were able to successfully test our device on both curved and flat labels and all scans were successful during the demonstration. One interesting problem we overcame was working with the OCR. While we successfully got it to pull text after we pre-processed the raw images, the text was not exactly what we wanted to print on the label. For example, it would pick up the item name, along with the brand, ingredients, etc. To fix this, we added a large language model to our design. It took in the OCR text in addition to the size of the text, used the prompts we gave it, and was able to give a streamlined and shortened label.
This has been a fun and rewarding project for our team. We’ve learned many lessons that we will take with us into future projects.