{"id":161,"date":"2024-03-23T09:21:19","date_gmt":"2024-03-23T14:21:19","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/?p=161"},"modified":"2024-03-23T09:21:27","modified_gmt":"2024-03-23T14:21:27","slug":"karens-status-report-for-3-23","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/2024\/03\/23\/karens-status-report-for-3-23\/","title":{"rendered":"Karen’s Status Report for 3\/23"},"content":{"rendered":"

This week I wrapped up the <\/span>phone pick-up detection implementation<\/span><\/a>. I completed another round of training of the YOLOv8 phone object detector, using over 1000 annotated images that I collected myself. This round of data contained more colors of phones and orientations of phones, making the detector more robust. I also integrated MediaPipe\u2019s hand landmarker into the phone pick-up detector. By comparing the location of the phone detected and the hand over a series of frames, we can ensure that the phone detected is actually in the user\u2019s hand. This further increases the robustness of the phone detection.<\/span><\/p>\n

After this, I began working on facial recognition more. This is to ensure that the program is actually analyzing the user\u2019s facial features and not someone else\u2019s face in the frame. It will also ensure that it is actually the user working and that they did not replace themselves with another person to complete the work session for them.<\/span><\/p>\n

I had first found a simple Python <\/span>face recognition library<\/span><\/a>, which I did some initial testing of. Although it has a very simple and usable interface, I realized the performance was not sufficient as it had too many false positives. Here you can see it identifies two people as \u201cKaren\u201d when only one of them is actually Karen.<\/span><\/p>\n

\"\"<\/p>\n

I then looked into another Python face recognition library called <\/span>DeepFace<\/span><\/a>. This has a more complex interface, but provides much more customizability as it contains various different models that can be used for face detection and recognition. I did some extensive experimentation and research of the different model options for performance and speed, and have landed on using <\/span>Fast-MTCNN<\/span><\/a> for facial detection and <\/span>SFace<\/span><\/a> for facial recognition.<\/span><\/p>\n

Here you can see the results of my tests for speed for each model:<\/span><\/p>\n

\u276f python3 evaluate_models.py\r\n24-03-21 13:19:19 - Time taken for predictions with VGG-Face: 0.7759 seconds\r\n24-03-21 13:19:20 - Time taken for predictions with Facenet: 0.5508 seconds\r\n24-03-21 13:19:22 - Time taken for predictions with Facenet512: 0.5161 seconds\r\n24-03-21 13:19:22 - Time taken for predictions with OpenFace: 0.3438 seconds\r\n24-03-21 13:19:24 - Time taken for predictions with ArcFace: 0.5124 seconds\r\n24-03-21 13:19:24 - Time taken for predictions with Dlib: 0.2902 seconds\r\n24-03-21 13:19:24 - Time taken for predictions with SFace: 0.2892 seconds\r\n24-03-21 13:19:26 - Time taken for predictions with GhostFaceNet: 0.4941 seconds<\/pre>\n
Here are some screenshots of tests I ran for performance and speed on different face detectors.<\/span><\/p>\n
OpenCV face detector (poor performance):<\/span><\/p>\n
\"\"<\/p>\n
Fast-MTCNN face detector (better performance):<\/span><\/p>\n
\"\"<\/p>\n
Here is an outline of the overall implementation I would like to follow:<\/span><\/p>\n