Category: Jong Woo’s Status Reports

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours): 

         This week, all the clamps for our apparatus have been properly replaced with screws and glues. Final height adjustments for initial cropping have been tweaked correspondingly, and our team started crafting the final paper as well. Our team decided to create a poster to go along with the in-person demo, and I did some preps for the in-person demo presentation. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?:

         Pre-processing has been fully implemented to its original scope. 

What deliverables do you hope to complete in the next week?:

        Final report will be completed along with the in-person demo.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours): 

         This week, our team’s testing apparatus has been built in the TechSpark’s Woodshop and pre-processing has been fully implemented to its original scope. More specifically, further accuracy was attained through the non_maximum_suppression’s capability to locate individual coordinates and get rid of most of the pre-existing redundancies in green dots drawn on top. Therefore, when an image is taken, it is properly pre-processed, and then cropped to 150 ~ 800 individual braille images, and then handed over to the ML classification pipeline. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?:

             Pre-processing has been fully implemented to its original scope. 

What deliverables do you hope to complete in the next week?:

         Final modifications and tweaking will be placed while prepping for the in-person demo. 

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours): 

         Currently ,I am primarily relying on the stats values (left, top coordinates of individual braille dots,as well as the width and height of the neighboring dots) from “cv2.connectedComponentsWithStats()” function. I have checked the exact pixel locations of the spitted out matrices and the original image and have confirmed that the values are in fact accurate. My current redundancies of dots come from the inevitable flaw of the connectedComponentsWithStats() function, and I need to get rid of the redundant dots sporadically distributed in nearby locations using the non_max_suppression. There is a little issue going on, and I do not want to write the whole function myself so I am looking for ways to fix this, but as long as this gets done, I am nearly done with the pre-processing procedures. Currently, I am looking more into how nms filter gets rid of redundant images and am planning on writing the nms filter function myself if it could not be debugged in time. I wasn’t able to make progress as of now (saturday) as far as the non_maximum_suppression filter is concerned, so further works are required tmrw in order to discuss some progress. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?:

             The current, pre-defined non_max_suppression is currently not fully functional and am currently in the process of debugging to properly get rid of redundant dots. However, once this is taken care of, the pre-processing subsystem is nearly done so I am hoping to get it solved and done by thanksgiving-ish. 

What deliverables do you hope to complete in the next week?:

         The current non_max_suppression filter would either be tweaked or re-written to properly process redundant dots sporadically present near the target location. Furthermore, I have currently confirmed that the center coordinates of each braille dots as well as their corresponding width and height are accurate given the direct pixel-by-pixel comparison with the original image, so cropping should be completed with relative ease as well. 

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours): 

This week, I gave a finishing touch to the current pre-processing filters that will be used to train our ML model. For thresholding, otsu thresholding, median thresholding, and gaussian thresholding with various threshold boundaries have been investigated. Because the thresholded image will then be eroded (to reduce small noises, by shrinking the currently existing dots) and dilated (extended to fill up the spaces to create a more wholesome circle), parameters have been tweaked multiple times and then fed into the erosion and dilation process, and individual results have been visually compared to opt for the better pre-processing results. For now, gaussian adaptive threshold with upper and lower boundaries as 21,4 exhibit preliminary thresholding. Below is the image of various different thresholding parameters and their corresponding result images. 

(please zoom in using ctrl + / –  to view the images here and below)

Similarly, canny edge filters, erosion, and dilations were all tested with various parameters to reach reasonable pre-processing results. Below is the code and corresponding comparison image that also includes the original braille image (img) as well as the final processed images(final1,2). 

         Furthermore, camera is integrated this week, and due to the resolution or lighting variations, the masking filters will need to be tweaked correspondingly to continuously produce reasonable pre-processing results. Below are the initially captured images with various colored contrast. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?:

             Progress is on schedule, and the upcoming week will be focused primarily on finishing vertical and horizontal segmentations that would lead into final cropping. 

What deliverables do you hope to complete in the next week?:

I hope to refine and finish the current horizontal segmentation, finish the remaining vertical segmentation, and lead into cropping.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours): 

         This past week, I focused on further applying the canny edge detection filter and using non-maximum suppression to draw out circles on top of the pre-existing braille characters. Canny edge detection, supported by OpenCV, allows various structural information to be extracted from an image while dramatically reducing the amount of the data to be processed. The results of the canny edge filter applied image will be facilitated for the non-maximum suppression, which would select a single entity (dot) out of many overlapping entities and draw out individual colored circles on the top. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?:

             Progress is on schedule, and the upcoming week will be primarily focused on the group work regarding camera integration. 

What deliverables do you hope to complete in the next week?:

        Camera Integration is the primary goal to be completed by next week. This is quintessential for the upcoming interim demo on the Nov 16th, and I believe that as long as camera integration could be completed in time, our team would be able to have some tangible deliverable by the interim demo date. 

Note: This weekly status report covers any work performed during the week of 10/15 as well as Fall Break

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours): 

         The week before mid-sem break was primarily invested in the crafting and review of the design report. Our team was able to efficiently divide the work loads for the design report based on the subsystems that we were individually working on, and have a thorough and considerate revision to finalize our design report.

          I have spent some time this week, looking into how the vertically and horizontally segmented images would be cropped and stored into a folder of separate file images. Since OpenCV had no specific functions for cropping, “Numpy array slicing” method will be adopted. Since every image files that has been pre-processed can be read in and stored as 2D array for the individual color channels, pixel values of the height and width of the area can be specified to crop specific regions of the images. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?:

             Progress is on schedule, and given how this past week was a mid-sem break, our team will focus on getting things back rolling and combine the parallelized works, getting ready for the interim demo that is approaching.

What deliverables do you hope to complete in the next week?:

                  Implementation of the canny edge filters along with the non-maximum suppression method for the ultimate edge contrasts will be done. 

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week, using various openCV functions that grayscale an image, or reduce the noise and enhance the edge contrast, or binarize an image, I have tested out various outcomes of applying different filters with controlled variables to attain a clean pre-processed image of braille text that would facilitate the recognition of each braille alphabets by our recognition ML dataset. For example, running the following python code below would display various potential intermediate outcomes of applying filters from the original braille image. 

After the initial image pre-processing stage, the image then needs to be segmented both vertically and horizontally, then save individually cropped braille alphabets into separate folder containing continuous jpegs of individual crops to be handled by the recognition ML dataset to be translated to the corresponding English Alphabet. This week, I worked on the vertical segmentation, and similar works will be applied for horizontal segmentation and the cropping and saving in coming weeks. 

In order to parallelize the workflow of our team, I manually cropped out the first 5 or 7 braille alphabets of the various versions of pre-processed images to be handed to Kevin for the next step of our algorithm, recognition phase. Kevin will then test the recognition ML algorithm and give metrics on the accuracy of translation given various clarities of pre-processed images. It is then going to be my goal to continuously enhance image-processing to match the thresholds required by Kevin’s metrics requirements. 

Last but not least, further method to enhance the accuracy rate from recognition phase was investigated, which was using non-max suppression from imults libraries. Since colored circles are drawn on top of the currently existing braille dots, there is a high chance that this way of pre-processing and cropping individual braille alphabets may return relatively higher accuracy rate. And the code for non-max suppression will be written from scratch in coming weeks. 

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

Things that were due this week were i) collaborative initial hardware boot ii) assess tradeoffs for various image pre-processing and segmentation methods and start testing out the effectiveness of each pre-processing algorithm, and iii) start writing some code from scratch for pre-processing. All goals were met and my progress is currently on schedule. 

• What deliverables do you hope to complete in the next week?

By the end of next week I plan on accomplishing the following: 1) Starting works on vertical & horizontal segmentation of the currently pre-processed images to attain cropped individual jpgs of braille alphabets, and 2) Research on non-max suppression methods using imults libraries to draw colored circles on top of pre-processed images for potential boost in recognition accuracy.