• Home
• Progress
• Schedule

WriteMe Weekly Blogs

March 4th - March 11th

Connor: Connor worked on our project website, the design document, and did research on our competitors. He wrote the project description and related work sections on the document and launched the first version of our website. We confirmed that that our competitors focus on recognizing characters on smart devices whereas WriteMe will focus on transferring handwritting into editable text.

Nikhil: Nikhil worked on our project design document and did research on the recursive closing transform algorithm for word segmentation. He learned that the algorithm worked by blurring the words, getting rid of the surrounding noise, and then uses a threshold to determine if a given region contains a word. Additionally, he wrote the architecture, design trade studies, and system description sections on the design document and added the schedule to the website.

Matt: Matt worked on the design document and did research on the character segmentation and recognition convolutional nueral network. He learned that it is a deep convolutional nueral network that outputs a guess at what a full word is. Additionally, he wrote the design requirements, design trade studies, project management, and references for the document.

March 12th - March 25th

Connor: This past week I worked on setting up the initial environment for the Android app. I did a ot of research on Android development in general and familiarized myself with Android Studio. I configured the app to take photos and store the image bitmap in a file. The next step is to transfer the bitmap to the AWS server for word segmentation.

Nikhil: This past week I worked on gaining a deep understanding of the recursive closing transform for the word segmentation. I have researched about different computer vision techniques for the segmentation, and have moved away from deep learning for this task as access to ground truth information is hard to attain. This removes the possibility of using supervised learning algorithms as creating a suitable objective function is not possible. For this coming week I am going to start implementing the segmentation algorithm and plan on having a rough implementation of the algorithm which shows promising results. This algorithm will use image normalization followed by a connected component analysis to trace words in the document.

Matt: This past week I worked on getting all of the dependancies up and running on the Microsoft azure server. As well as installing the CNN from GitHub and running initial tests on the different given weights. For next week I plan to experiment with different image input sizes and clean up the processing pipeline for speedier use once our preprocessing steps have been finished.

March 25th - April 4th

Connor: This past week I worked on linking the app to the AWS server. On the app, I used the image bitmap of the taken photo and sent it as a PNG file to the server. I configured the app so that when it boots it automatically establishes a connection with our server. I did this using some of the built in AWS APIs. PNG files are theoretically supposed to be lossless but I was still running into trouble with quality loss with the compressed bitmap.

Nikhil This past week I worked on getting the word segmentation to also keep the order of the words intact. I detected word lines by using y-coordinate buckets to keep track of words on a similar horizontal plane as other words in a given line. I also worked on make the thresholds and constant variables in the segmentation algorithm a factor of the scale of the image, the area of the non word regions to the word regions in the image and average intensity of the image. I have also gotten matlab to read the image from the s3 server and output the results into a results directory with each image stored as a seperate jpg file with each image labeled as the location of the word within the image. (nth word labeled as word_crop_#n.jpg).

Matt: This week I've shifted my focus to training a neural network on the emnist dataset for letters. We think that it will be easier to segment out individual letters and then regocnize them over a network for an entire word. After training I am going to begin working on the CV needed to align the images to a top down view.

April 4th - April 12th

Connor: This past week I worked on preserving the image quality when it gets sent to the AWS server. Additionally, I worked on finding a python API that can take the output of our CNN and load the individual words into a word document. Currently, we are experimenting with python-docx, a python library for creating and updating Microsoft Word.

Nikhil This past week I worked on trying to robustify my word segmentation algorithm, and started to work on the letter segmentation algorithm. The method I plan to use for the letter segmentation is similar to that of the word segmentation, but varying some parameters. I am also working on trying to port our code to the aws server. Matthew This week I worked on training a deep convolutional neural network to perform letter classification. Working on the EMNIST dataset I trained the network on 26 classes (lower and capital are in the same class). Achieving a validation accuracy of approximately 93.9%. I also worked on making the document from the letter predictions.