This past week I retrained the model several times (15+ times) to optimize our neural network and improve the accuracy the inference. I initially hit an accuracy of 65% and tried several things to improve the accuracy.

First, I added another convolution layer, and it improved the accuracy of our model to 75%. However, adding a second convolution layer made our model overfit. In other words, our model hit a 90% training accuracy, but remained at a 75% validation accuracy. Adding a third convolution layer actually lowered the overall accuracy, likely because our data set is not large enough.

Afterwards, I looked into adding regularization functions to improve our validation accuracy. Regularization functions are supposed to improve overfit models by making the model less responsive to noise. These were already built in to the Tensorflow library, so utilizing them was not difficult. I tried three different regularization functions (the sum of absolute values, sum of squared values, and the sum of both), and applied them to various parameters of the neural network – the bias, kernal, and activity variables. After training several times, I discovered that accuracy was significantly worse for some models, and approximately the same for others.

The only other solution I could think of was to enlarge our data set. From what I’ve looked up online, accuracy performance peaks using a data set of 1000-2000 images per class. Our data set currently only has 500 images each for raccoons, squirrels, and the lower body of humans. Over the next week I’ll scrape some images off of Google to enlarge our data set. Enlarging the data set also means that we can increase the number of parameters our neural network can program. This means we can expand our neural network to work with more layers, which may improve the accuracy.

Before I do that next week, I attempted to make even the number of images per class to see if that would change the results. Because I have over 12000 images each of dogs and cats, I removed 11500 of them from each set, so that each class could have 500 images. After training again, I discovered that the results were approximately the same (75% validation accuracy).

Finally, I played around with the batch size of our neural network. The batch size is a parameter which tells the neural network how many images to process at one step. Initially, this value was set to 32, but the smaller I made it, the more accurate our inference was. When I tuned the batch size down to a size of 8, I was able to get a validation accuracy of 78%. This was the best I could get it up to.

This week I began running my machine learning code on AWS. I first requested an EC2 instance with a GPU, which took about 2 days for AWS to process the request. Then on Wednesday I uploaded my data set and code to the EC2. Fortunately, the EC2 instance I requested was built for using Deep Learning libraries such as Tensorflow, so running the code on the GPU was a piece of cake. I simply had to change some settings on the EC2, and the code ran itself on the GPU (or so that’s what the terminal output said).

My first iteration of training resulted in a training accuracy of 76% and a validation accuracy of 74%, and took around 6 hours to run. This was surprisingly good. Although it doesn’t meet our goal of 95%, it seems reliable enough for demoing purposes. In order to bump up the accuracy, I added another convolution and activation (relu) layer. After training for a second time, I achieved a training accuracy of 95%, and a validation accuracy of 76%. What this means is that the model is overfit, and needs a larger data set and a regularization function. I will make the data set larger by flipping all of the images over the y-axis, and add a regularization function, and retrain tonight.

I will also draw a diagram for the solenoid sometime today. Once I finish the diagram, and once I retrain, I will update this post with the results.

This week I began learning how to use Tensorflow to build a Convolutional Neural Network and train a machine learning model. I did a tutorial online and found that it was surprisingly easy and intuitive.

Originally, the convolutional neural network was supposed to classify between 6 different types of objects: Cat, Dog, Squirrel, Raccoon, Legs, and Shoes. After taking sample images using a camera that was set up at the cat door height from one meter away, I realized that almost all of the time, the camera would see a human’s entire lower body, from the hip to the feet. Therefore, instead of classifying Legs and Shoes, we will get rid of both of them and classify Lower Body. I had previously found a large image data set of humans which contained annotated human body parts, so I wrote a python script to parse through some 10 gigabytes of images and only keep the ones which contained a human’s lower body. Then, I properly arranged each set of images into a directory for the Tensforflow script to read.

Now that the Tensorflow script has been written, the convolutional neural network built, and the data set prepared, I will test on a small sub-data set to ensure that the script is correct and then deploy the entire thing to AWS. I will slowly get to these two tasks over Spring Break and have them done before school starts again.