This week, I successfully added a hidden layer to the neural network. I worked with Mohini to gain a solid understanding of the back propagation algorithm. This gave me a clear understanding of how to update the three parameters – alpha, alpha2, and beta – of the new neural network. Alpha being the one between the input layer and first hidden layer, alpha2 being the one between the first and second hidden layer, and beta being the one between the second hidden layer and output layer. Using this knowledge, I incorporated the second hidden layer. In this algorithm, the two hidden layers have the same number of hidden units depending on the user input.
After this, I began testing the algorithm with our current training data to gauge its accuracy as compared to the algorithm with one hidden layer. The training data and testing data with the new neural network have an accuracy of 83.5% and 86.3% respectively, while the training and testing data with the previous neural network have an accuracy of 1.5% and 77.2% respectively. The results of the training data were as expected because an additional hidden layer reduces overfitting. The would result in a higher training error since the algorithm will classify the training data less correctly. However, I was disappointed to see the testing error slightly increase. I did not expect it to improve significantly, but at least marginally. This is because additional hidden layers bring more non-linearity to the neural network, and thus the classification. Therefore, it would be able to generalize the data better.
My next steps will be to continue to improve the accuracy of the neural network. I will modify the variable values such as the number of training data and the number of hidden units in each hidden layer. I will continue to monitor how these changes affect the accuracy of the algorithm and try to find the optimal parameters to achieve the highest accuracy. Additionally, I will start working on the web application again to complete the profile page.