Team CA: Intelligent Attendance and Participation Monitoring

• What did you personally accomplish this week on the project?

This was a short week for me since I have been traveling since Wednesday. This week I continued my work on the facial detection code. I have finished coding the Ada-boost classifier that works decently well. I am getting 95% accuracy for Faces, and 50% accuracy for Non-faces. Currently the code is only able to detect a single face in a 24×24 image. The next steps will be to write the sliding window code so that multiple faces can be detected in larger images. I used https://github.com/Simon-Hohberg/Viola-Jones to help guide my implementation.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I think I am on schedule to fully implement a working version of the facial detection by the first week after spring break.

• What deliverables do you hope to complete in the next week?

As mentioned earlier, I have written a classifier that works with images containing a single face. I will now work on the sliding window code to detect multiple faces in a larger image. Once the sliding window is complete, I will continue to optimize to get better accuracy and reduce the number of false-positives.

One significant risk the group face is not being able to increase our recognition accuracy to the required MVP levels. Local PCA was not successful, so we switched to Global PCA. That still was not good enough, so we implemented LDA, used more data, and more eigenvectors. Still now, we do not have enough accuracy. We are hoping and hypothesizing that we made a mistake with the fisherfaces implementation, which when fixed will increase our accuracy.

Another risk is that we need to come up with an algorithm to delineate between a stranger and non-stranger. Two approaches have already been field with limited success.

No major changes have been made to the existing design, and the schedule does not need to be updated from last week.

• What did you personally accomplish this week on the project?

This week I continued my work on the facial detection code. Initially, I was attempting to get the facial detection to work using openCV’s pre-trained weights, however, I found that all the images were getting rejected within the first few stages. Instead of spending too much time on this, I decided to move on and go back to writing my own AdaBoost classifier. Most of the code has been written. I also spent a decent amount of time working on the Design Review Report as well as preparation for my presentation. Later in the week, I worked on incorporating feedback from the presentation into the Design Review. In particular, I expanded on our methods for hand-raised detection as the level of detail for the presentation was not sufficient.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I think I am on schedule to fully implement a working version of the facial detection by spring break.  I was hoping to have more tangible results by now, but I think I am really close. The code has all been written and at this point I am debugging and stepping through my code to make sure I am following the algorithm properly. I hope to have a demo-able version my the meeting on Monday (3/4/2019).

• What deliverables do you hope to complete in the next week?

I will continue working on the facial detection code. I need to have a working version (that has >50% accuracy) by early next week, so that I work on optimizing and improving the detection.

What did you personally accomplish this week on the project?

This week I worked on researching and implementing eye alignment code that is responsible for making sure the image is aligned correctly. I am doing this using the following tutorial: https://www.pyimagesearch.com/2017/05/22/face-alignment-with-opencv-and-python/

The eye detection code is implemented using opencv and dlib. The algorithm here essentially finds the positions of the eyes and then computes the algorithm from the horizontal and position from the center, from which we obtain the affine transformation matrix that we can use to rotate the image in OpenCV. I am currently working on this code and I will commit this code to the group github repository around Sunday. I have also spent time reviewing the current PCA and LDA code in the repository to find out where we can obtain accuracy improvements as the results currently suffer from overfitting.

We also had to spend time on the Design Review Report as well as preparation for the presentation.

Is your progress on schedule or behind?  What deliverables do you hope to complete in the next week?

This week I am on schedule assuming the working code is committed on Sunday (3/4/19). For the next week I will be committing all of my time to working on the PCA and LDA code in the repository with Omar, with the goal of improving accuracy results. Deliverables for the next week would be more commits to the github repo with improved accuracy over this week. Hopefully the eye alignment code can help to improve the accuracy of the current result.

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I went over-time in the amount of work I did. I was hoping to put sufficient work to cover not only the expectations this week but also for the next two weeks. My work is summarized below.

• • • Finished implementation Local PCA. Resulted were poor. This led to the developments in the next points.
    • Completed implementation of Global PCA. Results were better but not good enough. Used library version of K-means to cluster. Implemented classes to recognize arbitrary image using the clustering.
    • Implemented a script that uses identifies faces in real-time (combines my work related to Global PCA, LDA, Fisherfaces).

 

• • • Completed LDA and    Fisherfaces implementation. Our implementation improved our training and testing accuracy by about 10% (to about 45% per class across 3 classes). However, we are still not getting the performance we need. We believe we are not correctly implementing LDA as our clustering is not exhibiting the traits that LDA is characteristic of (see below clustering with 3 classes projected into a C – 1 = 2-dimensional space).
    • 

• • • Developed algorithms to delineate between a stranger and non-stranger. I devised two methods for detecting strangers.

1. Use an images E2 K-means distance. This distance is its distance to the nearest centroid. If this distance is larger than some threshold we consider the image as that of a stranger. We decide the threshold value by taking it as the E2 K-means distance of the training data (e.g. 75% of training data images had E2 distance <= 500 so the threshold is 500)
2. Similar threshold to before except we use the reconstruction mean-squared error as a metric.

This method seemed promising but it did not work out. Below is a chart summarizing the source of inaccuracies we detected in our training and testing. The False Positive part of the stacked bar represent errors where our k-means correctly predicted the identity but because of our thresholding we designated that image as a stranger.

In order to understand why, I visualized how the E2 and MSE metrics for training data and stranger data compared (below). As you can see, my implicit hypothesis was that stranger images would have larger E2 distances and larger MSE’s than the training data. This was not the case, unfortunately. I am still working on a solution to the problem of detecting strangers.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • • On schedule.
• What deliverables do you hope to complete in the next week?
  • Since I accomplished much more than what I had to this week, I may take it easy for the next two weeks. The work I have been doing has been exciting so I may make progress anyways.

 

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I finished implementing Local PCA. The program takes in an arbitrary number of images, classes, and allows variable settings for eigenvectors chosen. It shows the reconstructions of all input images and also shows the k-means clustering results in a 3D plot. The clustering was not very good. After speaking with the professor, we decided we needed either more data, more eigenvectors, or to likely use LDA. We also started working on using dlib library to understand where the eyes in an image are so we can transform the image to get a frontal view.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • • On schedule.
• What deliverables do you hope to complete in the next week?
  • Improve our PCA performance. Implement or use library K-mean to cluster points and take a step closer to performing the actual recognition.

Design Review ppt

Proposal Presentation_ Team CA

Name: Kevan Dodhia

• What did you personally accomplish this week on the project?

This week I implemented a facial detection algorithm that uses openCV’s pre-trained Haar Cascades. While it is not working completely yet (there are some bugs), the code is able to correctly read the Haar Cascade XML file provided by openCV, pre-process the images and calculate the integral image (and sum regions).

I also spent a decent amount of time researching possible approaches for the raised-hand detection in preparation for the design review. While there were various purely vision based approaches that looked interesting, I found, read and understood a paper that used a novel technique that we will attempt to implement:
https://ijeir.org/administrator/components/com_jresearch/files/publications/IJEIR_835_Final.pdf

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

The facial detection is taking longer than anticipated. The adaboost training and cascading code is rather complicated to write. However, I have made some progress and am understanding the algorithms much more. I should be able to fully implement and test them before spring break.

• What deliverables do you hope to complete in the next week?

I will continue working on the facial detection algorithms.  I should be able to fully implement the version that uses openCV’s pre-trained Haar Cascades in by mid-week, and then work on the adaboost training code. I am on track to finish the facial detection before spring break.

 

 

Welcome to Carnegie Mellon University: ECE Capstone Projects. This is your first post. Edit or delete it, then start blogging!