{"id":198,"date":"2024-11-15T21:21:19","date_gmt":"2024-11-15T21:21:19","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/?p=198"},"modified":"2024-11-16T18:57:26","modified_gmt":"2024-11-16T18:57:26","slug":"jessies-status-report-for-11-16","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/11\/15\/jessies-status-report-for-11-16\/","title":{"rendered":"Jessie&#8217;s Status Report for 11\/16"},"content":{"rendered":"<p><span style=\"font-weight: 400\">What I did this week:<\/span><\/p>\n<ul>\n<li style=\"list-style-type: none\">\n<ul>\n<li style=\"font-weight: 400\"><b>RPI MODEL AND TENSION CODE INTEGRATION<\/b><span style=\"font-weight: 400\">: Shaye mostly led the integration process.\u00a0 I moved the code for the model onto GitHub so Shaye could work with it. I then copied the integrated version of the code from Shaye\u2019s branch on GitHub. We then worked together to debug any errors.\u00a0 The initial integrated code can be found on the jessie_branch <a href=\"https:\/\/github.com\/mich-elle-xu\/backintune\/commit\/29ece0e8415e9ce642f7ef95ff464549fdcaba0a\">at the blaze model commit<\/a>.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I built off the integrated model and tension algorithm code Shaye and I worked on to add the live feedback buzzer feature. I copied my previously written buzzer code into [], then using feedback from Shaye\u2019s rudimentary tension algorithm, if tension was detected, I triggered the buzzer for 0.1 seconds. I noticed that the video feedback was laggy when the buzzer went off. I believe this is because to have the buzzer buzz for 0.1 seconds, it is turned on, then waits 0.1 seconds (time.sleep(0.1)), then is turned off; the time.sleep is a blocking process, which would cause the code to lag as I saw. To remedy this, I created a thread each time the buzzer buzzed; the video seemed much smoother after this change. The code can be found on the jessie_branch at the <a href=\"https:\/\/github.com\/mich-elle-xu\/backintune\/commit\/3c7dd9d12583ec84df52943b5dca2c2efc865a2f\">buzzing roughly works commit<\/a>. Video of the working code can be found here: <a href=\"https:\/\/drive.google.com\/file\/d\/1jA_3Sd2Z57bAgNAS1DLm2VFd9FsDfc_w\/view?usp=sharing\">https:\/\/drive.google.com\/file\/d\/1jA_3Sd2Z57bAgNAS1DLm2VFd9FsDfc_w\/view?usp=sharing<\/a>\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">In the video, you can see that when I move my hands, the buzzing stops, and when my hands are still the buzzing occurs. This is because we correlate less movement with more tension. In the future, I\u2019ll look into how to make the buzzer buzz less frequently when tension is detected.\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<ul>\n<li><b>RPI WEB APP INTEGRATION<span style=\"font-weight: 400\">: I worked with Danny to interface video recording and calibration with the web application.\u00a0<\/span><\/b>\n<ul>\n<li><b><span style=\"font-weight: 400\">To interface the video recording, I had ChatGPT generate some basic functions to start and stop video recording. These functions were mapped to buttons on the web app. I mostly had to work to ensure the paths of where the video was being stored were correct and that the videos were uniquely named. This code can be found <a href=\"https:\/\/github.com\/mich-elle-xu\/backintune\/blob\/jessie_branch\/accelerated_rpi\/vid_record\/record_video.py\">on the jessie_branch in accelerated_rpi\/vid_record\/record_video.py<\/a><\/span><\/b><\/li>\n<li><span style=\"font-weight: 400\">I also wrote some code to interface the mirror_cam.py code with Danny\u2019s web app buttons. The code consists of functions to start and stop the mirror_cam.py code. The code can be found on the<a href=\"https:\/\/github.com\/mich-elle-xu\/backintune\/blob\/jessie_branch\/accelerated_rpi\/LCD_Module_RPI_code\/RaspberryPi\/python\/call_mirror_cam.py\"> jessie_branch at accelerated_rpi\/LCD_Module_RPI_code\/RaspberryPi\/python\/call_mirror_cam.py<\/a>. In the future, the calibration process will be much more detailed than this, but we have yet to fully think it through.\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><span style=\"font-size: 1rem\">Schedule:<\/span><\/p>\n<p><span style=\"font-weight: 400\">I am slightly behind schedule as I wanted to have the system fully integrated by this week. These are the elements I am missing: 1. recording while live feedback is being provided 2. triggering the start of a live feedback session with a button. However, I\u2019m making good progress towards that and think it is very achievable for next week.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">This week I focused on writing code specifically for demonstration purposes with the upcoming demo in mind, rather than working on completing the full system integration\u2014 showing that the buzzer interfaces with the tension algorithm and model on the RPi, the calibration interfaces with the web application, and video generated on the RPi interfaces with the web application.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Additionally, I was unable to write the Google form for the tension ground truth since we are waiting for Dr. Dueck to send us a tension detection rubric that we want to include in the form. Dr. Dueck has been away at a workshop in Vermont this week and thus has been delayed in providing the tension rubric.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Next week\u2019s deliverables:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Make a Google form and spreadsheet to collect and organize ground truth data from Prof Dueck and her students.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Work with Danny to make the real recording with live feedback triggerable through a button on the web app.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Record video while the live feedback is occurring.\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Work with Danny to ensure this video is accessible on the web app.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><span style=\"font-weight: 400\">Less time-sensitive tasks:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Investigate the buzzer buzzing frequency<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Start brainstorming the calibration code<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Experiment with optimizing the webcam mirroring onto the display\u00a0<\/span><\/li>\n<\/ul>\n<p><b>Verification of System on RPi:<\/b><\/p>\n<p><b>FRAMERATE:<\/b><\/p>\n<p><span style=\"font-weight: 400\">We have already been able to verify the frame rate at which the model runs on the RPi, since a feature to output the frame rate is already included in the precompiled model we are using. The frame rate of the model is around 22 FPS with both hands included in the frame.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">To determine the frame rate at which our system is able to process data (it\u2019s possible some of our additional processing for tension detection could slow it down) I have 2 ideas. The first is to continue to use the precompiled model\u2019s feature to output the frame rate. Our code builds on top of the code used to run the precompiled model, so I believe any slowdown as a result of our processing will still be observed by the precompiled model\u2019s frame rate output feature. My second idea was to find the average frame rate by dividing the total number of frames by the total length of the video. Both of these methods should result in a good idea of the frame rate at which our system processes video; additionally, I can use these 2 methods to cross-check each other.<\/span><\/p>\n<p><span style=\"font-weight: 400\">I plan to find the framerate when there are:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">2 hands in the frame (2 hands in the frame slows down the framerate of the model compared to 1 hand in the frame)\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Various amounts of buzzing (does the buzzer code affect frame rate?)<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Various piece types (does the amount of movement in the frame have an effect?)<\/span><\/li>\n<\/ul>\n<p><b>LATENCY:<\/b><\/p>\n<p><span style=\"font-weight: 400\">To find the latency of the system\u2019s live feedback, I plan to subtract the time audio feedback (buzz) was given by the time tension was detected. To collect the time at which tension was detected, I plan to have the program print the times (sync-ed to the global clock) at which tension was detected. To collect the time at which live audio feedback was given, I plan to have another camera (with a microphone) that will also record the practice session; I will sync this second video with the global clock as well and mark down the (global) times at which audio feedback occurs. I will then match the times at which tension was detected with the times at which audio feedback was given to find the latency.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">If I find that the frame rate varies given the different situations previously mentioned, then I will also run this latency test for those situations to find the effect of various framerates on latency.\u00a0<\/span><\/p>\n<p><b>POST-PROCESSING TIME:<\/b><\/p>\n<p><span style=\"font-weight: 400\">My plan to verify the post-processing time is fairly simple. I plan to record the time it takes for videos of various lengths to post-process, specifically for 1-minute of video, 5-minutes of video, 10-minutes of video, and 30-minutes of video. I will start timing when the user is finished recording and stop timing once the video is finished processing. <\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>What I did this week: RPI MODEL AND TENSION CODE INTEGRATION: Shaye mostly led the integration process.\u00a0 I moved the code for the model onto GitHub so Shaye could work with it. I then copied the integrated version of the code from Shaye\u2019s branch on GitHub. We then worked together to debug any errors.\u00a0 The &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/11\/15\/jessies-status-report-for-11-16\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Jessie&#8217;s Status Report for 11\/16&#8221;<\/span><\/a><\/p>\n","protected":false},"author":936,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[11],"tags":[5,9],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/198"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/users\/936"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/comments?post=198"}],"version-history":[{"count":3,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/198\/revisions"}],"predecessor-version":[{"id":201,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/198\/revisions\/201"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/media?parent=198"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/categories?post=198"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/tags?post=198"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}