Posted on

Francesca’s Status Report for Dec 6

This week, I’ve accomplished a lot personally, which I am very proud of. I’ve been responsible for most of the software implementation for this project, and it is complete! We were able to achieve 99% OCR accuracy, which we stalled at for a bit. I did this by switching from Tesseract to EasyOCR, which has proven to be more accurate for images we take with the Raspberry Pi. We take the picture with a button press, which is then translated to text. This text is then fed to a LLM, which cleans any errors and highlights the most important text we want to print with braille. This works really, really well.

We were able to finalize and code the next steps, as well. The text string is then converted into a string of numbers, which I’ll detail here: say you would like to print the word “hi”. To print h, we need two columns. For the left column, this is a top dot, a middle dot, and no dot. We can represent this as 1, 1, 0, and can interpret this s binary, making it a decimal 6. We repeat for the right column of the h, and for any other letters (in this case, i). The ends up with the string 6224. Then we wrote the code to have the Raspberry Pi read this string and make servo moves that correspond to the corresponding numbered actuator movements.

We were also able to finalize the code to get the speaker up and running, so the device asks the user to confirm the word about to be printed before proceeding. I’m glad we got this to work, because for a long time we were wondering how to have the user interface work with the knowledge the user is blind.

We were running a little behind on the hardware still, so I’ve been helping Bella and Abby with some 3D printing and design testing. We’ve been trying some different printing wheel prototypes to get as accurate of text as possible. As we move into this week, we will complete this prior to our presentation (!!!) and video submission. As for the final report, we have internalized the feedback that was given to us for our previous paper, and are making vast improvements prior to Friday.

Posted on

Team Status Report for Dec 6

The biggest risk of our project right now is the inconsistent printing. The software is working correctly and the hardware pieces are moving as expected. We are working on machining different pin variations to overcome this problem, but multiple iterations have not worked. There is a possibility our final demo will not have the printing that we anticipated. For testing, we are placing everything in the final casing and will be getting our final weight for portability requirements. We are doing user testing for easy of use, we unfortunately will be doing this with non-visible impaired users, but used feedback from the school from the blind and the ODR throughout our process. We are continuing to work on the pins and will be able to get a final force test for them. Our OCR is working up to our standard of accuracy as we test it on physical objects in addition to taking pictures of labels we have pulled up on the screen of our laptops to use more variety. It is successfully working on curved objects.

Posted on

Bella Woodard’s Status Report for Dec 6

I finalized the mechanical design of the embosser head and the device casing, completed the CAD files, and 3D printed and assembled the parts.  Progress is behind because we had to revisit our embossing approach, which required redoing the original casing design. The last steps are to ensure we’re getting the expected physical printout. Next week we will focus on finishing the final report to get back on schedule.

Posted on

Abby’s Status Report for Dec 6

This week, we wrapped up our mechanical design and construction. The software side is working to our standards, one change we made was moving on from Tessaract to EasyOCR. This is a relatively quick change, all that needed to be done was download the new program and replace it in the script where Tessaract was used. While Tessararct worked up to our standard for the demo, we believe once we incorporated pictures on the Rasp. Pi camera, there were quality differences that were not visibly noticeable to us, but hindered the OCR. I worked on the final testing of the embosser. We are unfortunately having issues with inconsistent printing, but all the mechanisms are moving as we designed. There is a possibility the final demo will not be able to show as high quality prints as expected, but we are continuing to trouble shoot. We are excited for the demo and will be wrapping up this weekend.

Posted on

Francesca’s Status Report for Nov 22

This week, I worked with Abby and Bella to integrate the software set up in Python with the hardware running via Raspberry Pi. On Monday, I placed the final orders needed for the project, and we received them on Wednesday. This order included mini breadboards, buttons, a speaker, and a Raspberry Pi specific camera to be used for BrailleMate. The rest of the week into the weekend, Abby and I connected the camera to the Raspberry Pi and confirmed that it works! We were then able to update the Tesseract code– how it works is that the Raspberry Pi camera takes an image called “photo.jpg” and puts it in a folder. This will always be the only image in the folder, as with every new scan, it is rewritten. Tesseract processing is then run on the image, and it produces “translation.txt” which is the English text version of the image. We will then use my python script that I wrote a few weeks ago to process translation.txt into the pins needed to have the Braille printing.

We were also able to integrate button pressing mechanisms for having the servos run, which is what will do the printing, albiet with a bit more of a lag than desired. We are integrating the button pressing for taking the picture tonight, as well as the audio output to interact with the user.

Next week is Thanksgiving break, but we intend to continue our work next week. We are taking BrailleMate home with us and will continue to work via FaceTime. Something we are a bit behind on is the housing for our product so that it is attractive for the user. We intend to complete that portion after Thanksgiving Break.

Posted on

Team Status Report for November 22

This week, we received the final main order of components. All subcomponents are coming together; a risk right now is the ease of integration. That will be the focus of this week as we plan the optimal way to start pulling components together. Our design has not changed, and with all the pieces, we can move forward at a faster pace. We’ve been meeting more outside of class and will continue to do so until the end of the semester. With the code working on CMU’s campus, the team will be able to work more efficiently instead of asynchronously. This week, we will begin to put our final presentation together and set goals for what we will have completed by the presentation date.

Posted on

Abby’s Status Report for Nov 22

This week, we received our last round of purchases. I worked on editing some of the STL files for the pin selector to address issues from the demo. I also began joining parts more stably together with screws. Additionally, I figured out how to work with the Raspberry Pi with the CMU wifi that previously gave us issues. This week we will continue working on integration and final prints of the components. I’ll specifically focus on getting the final version of the pin selector working to pass on to be integrated with the mapping code. Additionally, I will print out the wheel that will be driving the tape through the device.  We are still a little behind because of the HAT delay, but see a clear path forward at the moment.

Posted on

Francesca Cain’s Status Report for November 15

This past week, I worked primarily on preparation for our Interim Demo, and compiling the software work I had completed thus far into our slide deck. I’ve also spent some time debugging the code I have written for translating English text to Braille. Admittedly, I unfortunately had the flu, so I did not fully achieve all of the Tesseract testing this week that I planned to accomplish. Next week, I will get back on track by conducting this testing and taking pictures of medicine bottles for Tesseract to “read”. Next week,  I plan to spend time ordering final parts for our product– that is, our camera, buttons, and speaker, and will spend time helping Abby and Bella to integrate these hardware pieces on a breadboard to work with the Raspberry Pi and the software aspects of our work. The buttons will help provide a tactile response and mechanism for blind users to confirm a photo being taken, and the speaker providing audio (not visual) feedback.

Posted on

Francesca Cain’s Status Report for November 8

This week, I was able to essentially finalize my work on the Tesseract for use with OCR. I have utilized hundreds of public, research grade documents and images, including ~200 from FUNSD (Form Understanding in Noisy Scanned Documents) and ~1.3k from Tobacco800 (complex document image processing test set from Illinois Institute of Technology), which seemed to have about a 95% accuracy rate. I suspect this is largely because stock Tesseract is trained mainly on printed text, so its models and language priors don’t match handwritten cursive, which is what is often in FUNSD and Tobacco800. When I tested it on the Open Food Facts dataset, which is much closer to our use case (printed, commercial packaging to be scanned) and closer to what Tesseract was trained on, the accuracy rate was closer to 99%, which is our intended goal.

I’ve also written the bulk of the code for mapping Braille dot patterns to actuator sequence, but still need to more comprehensively test this, using both written test cases and with the physical BrailleMate device.

My progress is on schedule, and I am excited for how the project has come together! In my group, I have been responsible for much of the software, which has been interesting to learn about. In coming weeks, I will need to keep testing Tesseract on physical bottles, not just online datasets. Now that Abby has largely set up the Raspberry Pi, I will also need to update the Braille dot pattern code to work with our hardware, rather than just my Python test cases.

Posted on

Bella Woodard’s Status Report for November 15

This week I created and 3-D printed our Embossing head. Additionally, I updated  the Fusion designs for the handle and encasing of our device.

Embossing head: 

Handle Grip and Encasing Design:

My progress is only slightly behind. I planned to order a battery this week, but didn’t get the chance.  Next week, I plan to order our battery supply, finalize the Fusion model for the device base on Monday,  and begin 3-D printing our handle. Additionally, I aim to order our labeling media, so we can begin to print braille text.