{"id":76,"date":"2024-02-17T21:45:28","date_gmt":"2024-02-17T21:45:28","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/?p=76"},"modified":"2024-02-17T21:47:24","modified_gmt":"2024-02-17T21:47:24","slug":"karens-status-report-for-2-17","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/2024\/02\/17\/karens-status-report-for-2-17\/","title":{"rendered":"Karen’s Status Report for 2\/17"},"content":{"rendered":"

This week I finished implementing yawning and <\/span>microsleep<\/span><\/a> detection. These behaviors will help understand a user\u2019s productivity during a work session. I used <\/span>this paper<\/span><\/a> as inspiration for how to detect yawning and microsleeps. I calculate the mouth and eye aspect ratios, which tell us how open or closed the mouth and eyes are. If the ratios exceed a certain threshold for a set amount of time, it will trigger a yawn or microsleep detection. I implemented this using MediaPipe\u2019s face landmark detection rather than Dlib as used in the paper because MediaPipe is reported to have higher accuracy and also provides more facial landmarks to work with.<\/span><\/p>\n

Calibration and determining an appropriate threshold to trigger a yawn or microsleep detection proved to be more difficult than expected. For the detector to work on all users with different eye and mouth shapes, I added a calibration step at the start of the program. It first measures the ratios on a neutral face. It then measures the ratios for when the user is yawning, and then the ratios for when the user\u2019s eyes are closed. This is used to determine the corresponding thresholds. I normalize the ratios by calculating a Z-score for each measurement. My implementation also ensures that the detectors are triggered once for each yawn and each instance of a microsleep regardless of their duration. After finishing the implementation, I spent some time organizing the detectors into individual modules so that the code could be refactored and understood more easily. The code with my most recent commit with yawning and microsleep detection can be accessed <\/span>here<\/span><\/a>.<\/span><\/p>\n

I began exploring options for head pose detection and will follow a similar approach to that proposed in <\/span>this paper<\/span><\/a>.<\/span><\/p>\n

Overall, I am on schedule and making good progress. In the coming week, I will finish implementing head pose estimation to track where the user\u2019s head is facing. This will help us track how long the user is looking at\/away from their computer screen, which can be correlated to their focus and productivity levels. If this is complete, I will look into and begin implementing object detection to detect phone pick-ups.<\/span><\/p>\n

Below is a screenshot of the yawning and microsleep detection program with some debugging messages to show the ratios and their thresholds.<\/span><\/p>\n

\"\"<\/p>\n

<\/div>\n","protected":false},"excerpt":{"rendered":"

This week I finished implementing yawning and microsleep detection. These behaviors will help understand a user\u2019s productivity during a work session. I used this paper as inspiration for how to detect yawning and microsleeps. I calculate the mouth and eye aspect ratios, which tell us how open or closed the mouth and eyes are. If … <\/p>\n

Continue reading “Karen’s Status Report for 2\/17”<\/span><\/a><\/p>\n","protected":false},"author":842,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[3],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/posts\/76"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/users\/842"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/comments?post=76"}],"version-history":[{"count":4,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/posts\/76\/revisions"}],"predecessor-version":[{"id":78,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/posts\/76\/revisions\/78"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/media?parent=76"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/categories?post=76"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s24-teame0\/wp-json\/wp\/v2\/tags?post=76"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}