For many deaf or hard of hearing (HOH) individuals, sign language is a faster and more efficient way to communicate than written text. Written text can be inaccessible to deaf individuals due to its speed and the difficulty of learning a written language without it’s phonetic component.Â
Our project aims to break down this communication barrier by developing an automated speech-to-text system to translate ASL to written English in real-time. We plan to do this using computer vision and machine learning to instantly transcribe a live video feed into written English text. An FPGA edge device detects ASL from a video feed input and passes the detected words to an AI model that converts ASL-syntax sentences into natural English. The output text will then be displayed, enabling efficient two-way communication.
Any deaf or hard of hearing individual who prefers communicating via ASL but interacts with many non-ASL users on a regular basis could benefit from the platform we plan to build.