{"id":168,"date":"2025-04-12T19:24:56","date_gmt":"2025-04-13T00:24:56","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/?p=168"},"modified":"2025-04-12T19:24:56","modified_gmt":"2025-04-13T00:24:56","slug":"diyas-status-report-for-04-12-2025","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/2025\/04\/12\/diyas-status-report-for-04-12-2025\/","title":{"rendered":"Diya’s Status Report for 04\/12\/2025"},"content":{"rendered":"

I have worked on the following this week:<\/p>\n

    \n
  1. I\u2019ve been ironing out the design details for the post-cooking analytics feature. Based on concerns raised during our last meeting especially around how we detect when a step is completed and how to compute time per step. I am thinking of a few options such as to reduce noise from accidental flicks we already debounce each gesture using a timer. Only gestures that persist for a minimum duration (e.g. more than 300ms) are treated as intentional.\u00a0If the user moves to the next step and then quickly goes back it\u2019s pretty much a\u00a0 signal that they may have skipped accidentally or were just reviewing the steps. In these cases, the step won\u2019t be marked as completed unless they revisit it and spend a reasonable amount of time. I\u2019ll implement logic that checks if a user advanced and did not return within a short window making that as a strong indicator the step was read and completed. Obviously there is still edge cases to consider for example,\n
      \n
    1. Time spent is low, but the user might still be genuinely done. To address this I was thinking of tracking per-user average dwell time. If a user consistently spends less time but doesn\u2019t flag confusion or goes back on steps, mark them as ‘advanced’. If a user\u00a0shows a gesture like thumbs up or never flags a step we would treat it as implicit confidence even with short duration.<\/li>\n
    2. Frequent back and forth or double checking.\u00a0User behavior might seem erratic even though they are genuinely following instructions. I was thinking for this i won’t log a step as completed until user either a) proceeds linearly and spends threshold time or b) returns and spends more time.\u00a0If a user elaborates or flags a step before skipping, we lower the confidence score but still log it as visited<\/li>\n
    3. user pauses cooking mid step for example when they are using an oven and long time spent doesn’t always mean engagement. As we gather more data from a user, we plan to develop a more personalized model that will combine the gesture recognition, time metrics and NLP analysis of flagged content.<\/li>\n
    4. \"\"<\/li>\n<\/ol>\n<\/li>\n
    5. I\u2019ve been working on integrating gesture recognition using the pi camera and mediapipe. The gesture classification pipeline runs entirely on thepi. Each frame from the live video feed is passed through the mediapipe model, which classifies gestures locally. Once a gesture is recognized, a debounce timer ensures it isn\u2019t falsely triggered. Valid gestures are mapped to predefined byte signals, and I\u2019m implementing the I2C communication such that the Pi (acting as the I2C master) writes the appropriate byte to the bus. The second Pi (I2C slave) reads this signal and triggers corresponding actions like “show ingredients”, “next step”, or “previous step”. This was very new to me since I have never worked with writing an I2C communication. This still has to be tested.<\/li>\n
    6. I’m also helping Charvi with debugging the web app\u2019s integration on the Pi. Currently, we’re facing issues where some images aren’t loading correctly and also a lot of git merge conflicts. I\u2019ll be helping primarily with this tomorrow.<\/li>\n<\/ol>\n","protected":false},"excerpt":{"rendered":"

      I have worked on the following this week: I\u2019ve been ironing out the design details for the post-cooking analytics feature. Based on concerns raised during our last meeting especially around how we detect when a step is completed and how to compute time per step. I am thinking of a few options such as to … Continue reading Diya’s Status Report for 04\/12\/2025<\/span><\/a><\/p>\n","protected":false},"author":1073,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[5],"tags":[],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/posts\/168"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/users\/1073"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/comments?post=168"}],"version-history":[{"count":1,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/posts\/168\/revisions"}],"predecessor-version":[{"id":170,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/posts\/168\/revisions\/170"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/media?parent=168"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/categories?post=168"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s25-teamb1\/wp-json\/wp\/v2\/tags?post=168"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}