{"id":159,"date":"2024-10-26T15:05:53","date_gmt":"2024-10-26T15:05:53","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/?p=159"},"modified":"2024-10-26T15:05:53","modified_gmt":"2024-10-26T15:05:53","slug":"jessies-status-report-for-10-26","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/10\/26\/jessies-status-report-for-10-26\/","title":{"rendered":"Jessie&#8217;s Status Report for 10\/26"},"content":{"rendered":"<p><strong>This week\u2019s tasks:<\/strong><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I started the week with the intention of putting the model onto the KV260. I downloaded the <a href=\"https:\/\/github.com\/vidursatija\/BlazePalm\">model<\/a> and <a href=\"https:\/\/jonathantompson.github.io\/NYU_Hand_Pose_Dataset.htm\">dataset<\/a> (for the quantization step)<\/span><span style=\"font-weight: 400\">\u00a0suggested by Shaye. I parsed through the dataset and selected a subset that I thought would be best for the quantization step. Note that the model we chose is not same the MediaPipe model Shaye has been using. This is because the Vitis AI workflow is only compatible with Pytorch and TensorFlow files while the MediaPipe model is a TFLite file. For now, we opted to use the Blaze model instead which is in the form of a PyTorch file, but planned to look into conversion scripts in the future if the accuracy of the Blaze model was too low.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I met with Varun to discuss our concerns and difficulties while using the KV260. We learned that using the KV260 is unfavorable to the <a href=\"https:\/\/www.raspberrypi.com\/products\/ai-kit\/\">accelerated Raspberry Pi<\/a> for our specific application.<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">The Raspberry Pi has a more powerful accelerator and processor. The\u00a0 RPi can do 13 tera-operations per second (TOPS) while the FPGA can do around 1 TOP.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">The Raspberry Pi is easier to work with because there are more resources online. We found <a href=\"https:\/\/www.hackster.io\/AlbertaBeef\/accelerating-the-mediapipe-models-on-raspberry-pi-5-ai-kit-1698fe\">someone using the RPi with our exact model application<\/a> who achieved a frame rate of 22-25 fps. The KV260 would take a lot of effort to reach our target frame rate (if it\u2019s even possible). Varun said that the frame rate of processing the Restnet 18 model (from the quickstart tutorial), which I found to be 15-24 FPS from doing the tutorial a previous week, is likely the maximum capability of the KV260.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Varun mentioned the following benefits of using the KV260; however, none of them apply to our project<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">The KV260 is more power efficient than the RPi. We plan to have users plug the device into wall power (no battery necessary), so this is not a concern for us.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">The KV260 has more I\/O. It has more pins that can be used for anything while the RPi just has a small amount of GPIO pins and some USB ports. However our application has a limited number of I\/O (display, buzzer, webcam, web app hosting RPi), so the additional I\/O pins that the KV260 provides are not necessary for us.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">The KV260 can be useful if there are multiple things that need to be accelerated. However, our application only requires the model to be accelerated. Varun believes that software should be fast enough to run the tension detection algorithm.\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I read through the <a href=\"https:\/\/www.hackster.io\/AlbertaBeef\/accelerating-the-mediapipe-models-on-raspberry-pi-5-ai-kit-1698fe\">documented RPi workflow for putting MediaPipe\u2019s hand landmark model onto the accelerated RPi<\/a> to prepare to do it myself next week.<\/span><span style=\"font-weight: 400\">\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">I also worked on the ethics assignment that was due this week.\u00a0<\/span><\/li>\n<\/ul>\n<p><strong>Schedule:<\/strong><\/p>\n<p><span style=\"font-weight: 400\">I failed to complete last week\u2019s assigned deliverable because we decided to shift from putting the model onto the KV260 to putting the model on an accelerated RPi. I\u2019m currently behind schedule as Shaye and I originally planned to finish the FPGA\/CV integration by the coming Wednesday; however, due to the pivot from KV260 to Raspberry Pi, our schedule has changed as noted in the team status report.\u00a0<\/span><\/p>\n<p><strong>Next week\u2019s deliverables:<\/strong><\/p>\n<p><span style=\"font-weight: 400\">Next week I plan to integrate the MediaPipe model onto the RPi. I should also have a rough idea of how to implement the buzzer and display feature. I will also look into how to set up UART to communicate with the web app hosting RPi.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>This week\u2019s tasks: I started the week with the intention of putting the model onto the KV260. I downloaded the model and dataset (for the quantization step)\u00a0suggested by Shaye. I parsed through the dataset and selected a subset that I thought would be best for the quantization step. Note that the model we chose is &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/10\/26\/jessies-status-report-for-10-26\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Jessie&#8217;s Status Report for 10\/26&#8221;<\/span><\/a><\/p>\n","protected":false},"author":936,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[11],"tags":[5,9],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/159"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/users\/936"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/comments?post=159"}],"version-history":[{"count":1,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/159\/revisions"}],"predecessor-version":[{"id":160,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/159\/revisions\/160"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/media?parent=159"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/categories?post=159"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/tags?post=159"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}