This week, I worked with Sid to achieve and verify our design requirements for classification accuracy and latency using the completed PCB.
I improved the convolutional neural network training by adding data augmentation to the training set. Since the camera occasionally shifts between the days, we added data augmentations that choose randomly-shifted crops of the input image to model those physical movements. I applied this data augmentation only to the training set.
Sid and I captured and labelled 2724 new captures with the LED’s from the PCB. Combined with the new data augmentation and a learning rate scheduler, we achieved the best validation accuracy of 99.1%. This model achieved 98.1% accuracy on the test set. This is the best validation accuracy we have achieved, so we are using it for the MVP.
We also measured the classification latency. The design requirements specify 2 second latency from capture to web update. We dealt 52 cards over a 52 second period and achieved the following latency statistics for each card:
min: 0.162s
mean: 0.184s
median: 0.177s
max: 0.224s
The maximum latency of 0.224s is far less than our design requirement of 2 seconds.
I have now completed the imaging and machine learning system. We have some final testing to carry out, but so far, we have achieved every design requirement that we have tested.