{"id":114,"date":"2024-09-28T22:20:00","date_gmt":"2024-09-28T22:20:00","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/?p=114"},"modified":"2024-09-28T22:20:43","modified_gmt":"2024-09-28T22:20:43","slug":"jessies-status-report-for-09-28","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/09\/28\/jessies-status-report-for-09-28\/","title":{"rendered":"Jessie’s Status Report for 09\/28"},"content":{"rendered":"

This week\u2019s tasks:<\/strong><\/p>\n

    \n
  1. After finding a webcam that we plan to use and talking to Jim Bain and Joshna more about the camera stand design, I finalized the necessary measurements. Since we have chosen a webcam, I used that webcam\u2019s FOV to calculate the distance between the camera and the keyboard. Additionally, I worked with the group to measure the distance between the camera and the base of the stand (how much does the stand have to bend over the player). The measurements and calculations can be found in the team status report. From these calculations, the stand will have to be very long (about 7.5\u2019 or 8\u2019 if we want some wiggle room). I\u2019m starting to look into attaching a gooseneck (various flexible plumbing materials) to a mic stand or tripod.\u00a0<\/span><\/li>\n
  2. We were able to acquire the KV260, so I stopped working on Varun\u2019s Vivado tutorial since I\u2019m not sure if it\u2019s still applicable. Instead, I started looking into Vitis AI.\u00a0<\/span>\n
      \n
    1. The KV260 came without any accessories, so we had to scavenge to find them. We were able to find all the necessary components but discovered the\u00a0<\/span><\/li>\n<\/ol>\n<\/li>\n
    2. I learned more about the flow of Vitis AI and what would be required from us. My findings are largely based on the Xilinx documentation\/tutorials but also from <\/span>https:\/\/github.com\/PeterQuinn396\/KV260-Pose-Commands\/tree\/main<\/span><\/a> and <\/span>https:\/\/www.hackster.io\/AlbertaBeef\/accelerating-the-mediapipe-models-with-vitis-ai-3-5-9a5594#toc-model-inspection-7<\/span><\/a> who successfully put a MediaPipe model on the KV260.<\/span>\n
        \n
      1. Vitis AI takes in models in either a TensorFlow or PyTorch format. However MediaPipe is in a TFLite format. The hackster post mentions conversion scripts from other people, but they had little success using these scripts. We might have to write a conversion script or look into finding another model that is in a compatible format.<\/span><\/li>\n
      2. Vitis AI first takes the model and inspects it. From the hackster post, even if the model doesn\u2019t pass the inspection, the Custom OP flow can be used as a workaround. Then Vitis AI will optimize the model (prune it). Then the model gets quantized. This step requires several 100s to 1000s of samples.\u00a0<\/span>\n
          \n
        1. Since we don\u2019t have access to the data set used to train the MediaPipe model, we will have to find a data set and convert it to samples. Both Peter Quinn and the author of the hackster post write scripts to convert data sets to samples that we can reference.\u00a0<\/span><\/li>\n
        2. A concern I have right now is the quality of the samples and how that might influence the accuracy of the model. The author of the hackster post experienced degraded model accuracy as a result of using bad samples. We will likely have to experiment with different data sets or sample generation scripts to maintain a high model accuracy.<\/span><\/li>\n<\/ol>\n<\/li>\n
        3. Vitis AI then compiles the mode. I realized I\u2019m not sure how to interact with the output of the model on the FPGA, so I will have to look into that in order to successfully calculate the kinematics on the FPGA.<\/span><\/li>\n<\/ol>\n<\/li>\n
        4. I worked with Danny to follow this quick guide from xilinx <\/span>https:\/\/xilinx.github.io\/Vitis-AI\/3.0\/html\/docs\/quickstart\/mpsoc.html<\/span><\/a>. We downloaded Vitis AI and ran the scripts, but had to stop at the setting up the target step since we discovered the power supply we found was not the right size for the board.<\/span><\/li>\n<\/ol>\n

          Schedule:<\/strong><\/p>\n

          I\u2019m still on schedule.<\/span><\/p>\n

          Next week\u2019s deliverables:<\/strong><\/p>\n

            \n
          1. Once the power supply comes in, successfully run Vitis AI on the example models provided.\u00a0<\/span><\/li>\n
          2. Either determine it\u2019s not possible\/feasible to run kinematic calculations on the FPGA, or have a plan for a way to execute it.\u00a0<\/span><\/li>\n
          3. Finalize plans for a camera stand so that we have a rudimentary stand the next time we meet with Prof Dueck\u2019s students on 10\/9.\u00a0<\/span><\/li>\n<\/ol>\n

             <\/p>\n","protected":false},"excerpt":{"rendered":"

            This week\u2019s tasks: After finding a webcam that we plan to use and talking to Jim Bain and Joshna more about the camera stand design, I finalized the necessary measurements. Since we have chosen a webcam, I used that webcam\u2019s FOV to calculate the distance between the camera and the keyboard. Additionally, I worked with … <\/p>\n

            Continue reading “Jessie’s Status Report for 09\/28”<\/span><\/a><\/p>\n","protected":false},"author":936,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[11],"tags":[5,9],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/114"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/users\/936"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/comments?post=114"}],"version-history":[{"count":2,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/114\/revisions"}],"predecessor-version":[{"id":116,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/114\/revisions\/116"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/media?parent=114"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/categories?post=114"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/tags?post=114"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}