{"id":181,"date":"2024-11-09T04:45:44","date_gmt":"2024-11-09T04:45:44","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/?p=181"},"modified":"2024-11-09T04:47:37","modified_gmt":"2024-11-09T04:47:37","slug":"jessies-status-report-for-11-09","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/11\/09\/jessies-status-report-for-11-09\/","title":{"rendered":"Jessie&#8217;s Status Report for 11\/09"},"content":{"rendered":"<p><span style=\"font-weight: 400\">This week\u2019s tasks:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><b>CONNECTING RPI TO CAMPUS WIFI<\/b><span style=\"font-weight: 400\">: At the beginning of the week, I successfully connected RPi to school wifi by registering it to CMU-DEVICE.<\/span><\/li>\n<li style=\"font-weight: 400\"><b>RUNNING PROGRAM AT BOOT<\/b><span style=\"font-weight: 400\">: I also investigated having a program run automatically at boot (so we won\u2019t need to ssh into the RPi each time we want to run a program). After tinkering around a bit, I was able to get a sample program that writes \u201chello world\u201d to a file to run automatically at boot. I wonder if a program that continuously loops (likely like the program we will end up writing) will be able to work in the same way. I can experiment more with it next week or with the integrated code.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">The next time I came to work on the project I was unable to boot the RPi. It seems like the OS couldn\u2019t be found on the RPi. When I tried to reflash the SD card, the SD card couldn\u2019t be detected, indicating that something happened to it; we suspect we broke it when we were putting the RPi into the case. We had another SD card on hand; however, it was a 16 GB SD card instead of a 128 GB one. I redid my work on the RPi with the 16 GB SD card (installing necessary programs and starting a program automatically at boot). This would have been fine had we not planned to put video files for testing purposes on the Pi. Therefore we will likely have to transfer the data on the 16 GB SD card to a different 128 GB SD card in the future.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><b>ACCELERATED RPI TUTORIAL<\/b><span style=\"font-weight: 400\">: I finished following the <a href=\"https:\/\/www.hackster.io\/AlbertaBeef\/accelerating-the-mediapipe-models-on-raspberry-pi-5-ai-kit-1698fe\">tutorial to put the hand landmark model onto the accelerated RPi<\/a><\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Overall it was pretty straightforward. It was difficult to attach the accelerator to the Pi at times (the Pi wasn\u2019t picking up that it was connected).\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I was stuck for a bit because there was no video output as the tutorial said should pop up. Danny helped me out and we figured out it was because I didn\u2019t have X-11 forwarding enabled when I ssh-ed into the Pi on my laptop. Once I had X-11 forwarding enabled, the video output was very laggy. As a sanity check, I re-ran the direct media pipe model on the Pi (no acceleration) like I did last week, and it had a much slower frame rate (~4 fps instead of the previously observed 10 fps). Danny also helped me figure out this because last week I used a monitor to output the video instead of X-11 forwarding. Once I connected the Pi to a monitor to output the video, I was able to achieve a frame rate of around 21-22 fps on the accelerated RPi. The video output causing a drop in frame rate should not be a concern for us as we don\u2019t care much about the video output for live feedback and only need to use the outputted landmark information (in the form of a vector) for our tension calculations.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\">Video of accelerated RPi model output:\u00a0https:\/\/drive.google.com\/file\/d\/1msm4iRN0igps-D62fNLJPaKeeLQqsShn\/view?usp=sharing<\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><b>ACTIVE BUZZER<\/b><span style=\"font-weight: 400\">: I had ChatGPT quickly write some new code for the newly acquired active buzzer. There are 2 versions of code on the GitHub repo that I tried, which output at different frequencies. On my branch (jessie_branch) <a href=\"https:\/\/github.com\/mich-elle-xu\/backintune\/blob\/jessie_branch\/accelerated_rpi\/active_buzzer.py\">active_buzzer.py<\/a> outputs at a higher frequency<\/span><span style=\"font-weight: 400\">\u00a0and <a href=\"https:\/\/github.com\/mich-elle-xu\/backintune\/blob\/jessie_branch\/accelerated_rpi\/active_buzzer_freq.py\">active_buzzer_freq.py<\/a> outputs at a lower frequency.<\/span><span style=\"font-weight: 400\">\u00a0We can tinker with this more at a later time\u2013 I think the high frequency can be very distracting and alarming.\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">higher frequency video:\u00a0https:\/\/drive.google.com\/file\/d\/10R0AOH2a84ZJaFh7ogOY7JOEHq7ynsl4\/view?usp=sharing<\/span><\/li>\n<li>lower frequency video:\u00a0https:\/\/drive.google.com\/file\/d\/1epzOV_M6fjuQC4USLHoPRA3CS1ay1ExG\/view?usp=sharing<\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><b>CALIBRATION DISPLAY<\/b><span style=\"font-weight: 400\">: The 2-inch display arrived this week! I tried to follow <a href=\"https:\/\/www.waveshare.com\/wiki\/2inch_LCD_Module#Python\">this spec page<\/a> to get the display set up:\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I hooked the display up to the RPi as the page indicated; however, I was unable to get the examples to work successfully. Danny was able to get the examples to work and more information can be found in his status report.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Once the examples were working I was able to work with ChatGPT to write some code to mirror the webcam output onto the display for the calibration step. The code can be found at <a href=\"https:\/\/github.com\/mich-elle-xu\/backintune\/blob\/jessie_branch\/accelerated_rpi\/LCD_Module_RPI_code\/RaspberryPi\/python\/mirror_cam.py\">backintune\/accelerated_rpi\/LCD_Module_RPI_code\/RaspberryPi\/python\/mirror_cam.py<\/a> on the jessie_branch.<\/span><span style=\"font-weight: 400\">\u00a0I had to edit their outputted code a bit to properly rotate the video output onto the display. The webcam output has a huge delay and low frame rate. We don\u2019t think this is a huge issue as the webcam mirror will only be used during the setup\/calibration step to help the user ensure their hands and the entirety of the piano are within the frame; therefore, a high frame rate is not necessary but could be frustrating to work with. If there is time in the future, I can look further into optimization possibilities.<\/span><\/li>\n<li style=\"font-weight: 400\">video of laggy display:\u00a0https:\/\/drive.google.com\/file\/d\/1eLImQROqo-vjqi8m00PNjzmGhWVeltTi\/view?usp=sharing<\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I also responded to AMD\u2019s Andrew Schmidt\u2019s email (and Prof Bain\u2019s Slack message) asking for feedback.\u00a0<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400\">Schedule:<\/span><\/p>\n<p><span style=\"font-weight: 400\">I am very much on schedule, even completing less time-sensitive tasks. At Joshna\u2019s suggestion, we decided to combine the web app hosting RPi and the accelerated RPi onto one Pi, therefore the previous UART interface is not necessary. Next week I\u2019m hoping to make a lot of progress with full system integration.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Next week\u2019s deliverables:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Make a Google form and spreadsheet to collect and organize ground truth data from Prof Dueck and her students.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Interface the output of the hand landmark model on the RPi with Shaye\u2019s code.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Work with Danny to interface the web app with the RPi. Specifically, try to get programs to run by clicking buttons on the web app.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Start looking into how to post-process video and transfer it to the web application.\u00a0<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400\">Less time-sensitive tasks:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Experiment with optimizing the webcam mirroring onto the display\u00a0<\/span><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>This week\u2019s tasks: CONNECTING RPI TO CAMPUS WIFI: At the beginning of the week, I successfully connected RPi to school wifi by registering it to CMU-DEVICE. RUNNING PROGRAM AT BOOT: I also investigated having a program run automatically at boot (so we won\u2019t need to ssh into the RPi each time we want to run &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/11\/09\/jessies-status-report-for-11-09\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Jessie&#8217;s Status Report for 11\/09&#8221;<\/span><\/a><\/p>\n","protected":false},"author":936,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[11],"tags":[5,9],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/181"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/users\/936"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/comments?post=181"}],"version-history":[{"count":2,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/181\/revisions"}],"predecessor-version":[{"id":183,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/181\/revisions\/183"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/media?parent=181"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/categories?post=181"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/tags?post=181"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}