Author: youngwo1

This week our team began to integrate the web application and the backend of the image processing and the classification together I started working on adapting the classifier to the Django app by importing the network weights from within the app. Meanwhile, I’m also working on hand detection which should lessen the amount of work on the image processing side. Admittedly, I am starting to get a little bit uneasy about the amount of work left to be done. It would have been nice to have a deliverable product by now, regardless of its level of sophistication, but we still have the training and the feature extraction left to be done. So I have begun to look into transfer learning, which involves taking a generic pretrained model for a similar task and supplements it with training under a more specific and personalized dataset. The amount of time it takes to implement this seems to be less than the training time it would save from learning from scratch. I am hoping these give us a more reasonable path towards reaching our benchmark results.

This week I updated the neural network to be a fully learning CNN. I’ve implemented a couple different CNNs, namely a mini CNN that takes in 28×28 pixel inputs and outputs a likelihood over the different possible classes, as well as the AlexNet that takes in larger image sizes and has more filters and larger convolutions with the same output dimensions. I’m having some syntax bugs with the SoftMax layer at the moment which are confusing me but as soon as that gets fixed, I have a sample data set to test out the miniCNN on and check that it can learn. As of now I’ve decided to use the same network and input dimensions for both static and dynamic images. Afterwards, I’ll begin to run the AlexNet on the ASLLVD or Boston RWTH set for static images and resume feature engineering modules.

This week was unexpectedly very exhausting and difficult for me all of a sudden, so I have not yet made progress since Monday, but I will work on the project tomorrow and make updates here accordingly.

I got more work done this week in designing the neural network for general inputs of size m by n. As of now we have a neural network without convolutional layers, with two Dense layers with ReLU activations, Adam optimizer, and Multinomial cross entropy loss. This is the base network we will use for both static signs and dynamic signs, although the specifics of the hyperparameters and the layers will have to differ based on the performance we observe on the two classes. I’ve implemented the skeleton of the training process using the tensorflow graph flow model and will spend the rest of the week figuring out adding convolutional layers and returning to the feature extraction step to begin the dynamic gesture classification process.

This week I went out of town for a few days without internet connection and was mostly occupied with fitting midterms and assignments for the rest, so I didn’t get too much done on my end. However, I have a pretty good sense on how to design the neural network so that will be my next task going forward. One concern I’m having is the tradeoff between writing  computer vision algorithms such as hand detection and using existing modules and their functions. Although I would rather write most of the functionality from scratch, I’m concerned that my implementation won’t beat the existing optimized module functions and cause performance to suffer, such as increasing the response time beyond the real time constraints.