I ended up spending a majority of Spring Break working on Capstone. It was not exactly expected but I enjoyed the programming and couldn’t get myself to stop.
Anyways, as a product of that work, I was able to write a working MVP program that accomplishes many of our MVP objectives by fusing together my work with facial recognition and hand detection.
That MVP script, using my webcam, does the following:
- detect faces (OpenCV). Hopefully, we can fuse Kevan’s work with detection to replace OpenCV’s here soon.
- recognize faces (my PCA + LDA = fisherfaces implementation from scratch) with < 1 second delay. Thanks to a novel boosting algorithm I implemented, the accuracy on our training set increased to 70-80% on average. However, accuracy in real-life webcam usage is lower because the faces are seen in new enviroments (tilt, brightness, etc). Hopefully, Neeraj’s work with dlib and face regularizatoin can help with this.
- Detects hands raised at the height of the persons head or higher with 1 second delay. Fairly high accuracy (~75% or greater). Built all from scratch (except k-means algo). Re-learns each individuals face pigments every 10 seconds (takes about ~2 seconds)
- Records all recognition and hand detection and displays it in charts.
- Runs a suite of automatic tests on the recognition portion of the script. Displays results in useful charts.
I can’t summarize all of the things I did over break. But I will touch on the important ones.
Novel Boosting Algo for Fisherfaces
Our fisherfaces implementation was not generating the accuracy on out testing set that we wanted (about ~55% with 4 classes). I came up with a boosting algo where several iterations of unique fisherfaces were run and their results fused. Each iteration differed on mainly two factors: Firstly, they learned on somewhat different training sets. Thus, each learned in a unique way, yet still saw most of the training data so that each did better than random guessing. Secondly, each weak learner used a random image format when doing PCA, LDA (options include gray, red, blue, green; hue was deemed ineffective). Each weak learner voted on classification.
I am currently far ahead of schedule.