{"id":111,"date":"2024-09-28T19:40:49","date_gmt":"2024-09-28T19:40:49","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/?p=111"},"modified":"2024-09-28T19:40:49","modified_gmt":"2024-09-28T19:40:49","slug":"team-status-report-for-09-28","status":"publish","type":"post","link":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/09\/28\/team-status-report-for-09-28\/","title":{"rendered":"Team Status Report for 09\/28"},"content":{"rendered":"<p><b>General accomplishments:<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We worked on the design report and slides.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Jessie looked into the AMD workflow for putting a model on the FPGA. We now have a more fleshed-out idea of the tasks that need to be done. More details can be found in Jessie\u2019s status report.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Danny finished the Django tutorial.\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We asked Dueck\u2019s students on their preferences on different WebApp features. More details can be found in Danny\u2019s status report.<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We met with Dueck\u2019s students to preliminarily test system\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Need to track angle variation rather than consistent angle position\u2014more details in Shaye\u2019s status report<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We have better specified the setup measurements and requirements based on additional measurements of Shaye playing piano. (Shaye not pictured \ud83d\ude41 )<\/span><\/li>\n<\/ul>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-full wp-image-112\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-content\/uploads\/sites\/331\/2024\/09\/Screenshot-2024-09-28-at-3.39.46\u202fPM.png\" alt=\"\" width=\"870\" height=\"770\" srcset=\"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-content\/uploads\/sites\/331\/2024\/09\/Screenshot-2024-09-28-at-3.39.46\u202fPM.png 870w, https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-content\/uploads\/sites\/331\/2024\/09\/Screenshot-2024-09-28-at-3.39.46\u202fPM-300x266.png 300w, https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-content\/uploads\/sites\/331\/2024\/09\/Screenshot-2024-09-28-at-3.39.46\u202fPM-768x680.png 768w\" sizes=\"(max-width: 767px) 89vw, (max-width: 1000px) 54vw, (max-width: 1071px) 543px, 580px\" \/><\/p>\n<p><b>Risks:\u00a0<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We are slightly concerned about converting the currently used MediaPipe model (TFLite) to PyTorch or TensorFlow, the only 2 formats compatible with Vitis AI. We plan to look into\u00a0<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">existing scripts that convert between the 2 formats or\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">find different models that are in the compatible format\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We are also worried about the MediaPipe model losing accuracy once compiled on the FPGA due to over-optimization and bad inputted samples during the quantization phase<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We plan to try multiple data sets and ways to convert samples to tweak accuracy<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We need to look into how to implement kinematics on FPGA. We are not sure how to interact with the output of the model<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Contingency: switch to RPi\u2014will request on 10\/8 if needed so we have time to work on it before fall break<\/span><\/li>\n<\/ul>\n<p><b>Updates on the system:<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">We figured out how to port model &amp; communicate between FPGA &amp; RPI<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Using Vitis AI workflow for model conversion\u00a0<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Using UART for board communication\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><b>Schedule updates:<\/b><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Still on schedule, no updates<\/span><\/li>\n<\/ul>\n<p><b>Additional questions:<\/b><\/p>\n<p><span style=\"font-weight: 400\">Part A is written by Jessie, Part B is written by Danny, and Part C is written by Shaye<\/span><\/p>\n<p><span style=\"font-weight: 400\">Public health\/ safety\/ welfare factors:<\/span><\/p>\n<p><span style=\"font-weight: 400\">Our product aims to reduce injury among piano players by identifying and correcting harmful playing positions. Injury among piano players is extremely common (50-70%), with many injuries being related to the hand and wrist. The effects of being injured can have mental health impacts additionally, as it may prevent players from being able to practice for extended periods. Our system will provide players with real-time and post-processed feedback on the player\u2019s hand position. This will help them correct their position, even without the guidance of a piano teacher. We realize that providing bad feedback can lead to more injury; thus, we\u2019re taking special care to focus on the testing and verification metrics we\u2019ve mentioned throughout this process to ensure system accuracy.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Social Factors:<\/span><\/p>\n<p><span style=\"font-weight: 400\">Our product will change the dynamic between the piano teacher and the student. Instead of having the piano teacher focus on the health of the student and harmful techniques, they can focus on music-related content. This product could also help encourage beginners to get into piano playing without the potential worry of becoming injured, lowering the barrier to entry. This could potentially lead to an increase in piano players.\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Economic factors:\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">Our project will be the first product pianists can use to monitor their technique while practicing. Thus, although the total cost of our current system is high (FPGA cost, camera, stand, etc), creating a proof of concept using more general hardware will allow for more projects down the line to decrease the cost and create more accessible &amp; commercialized products. With more specific hardware\/ boards dedicated to running our system, the cost will decrease, allowing the product to be available for all piano players. For now, even just parts of this tool (CV pipeline, video saving features) can help save pianists from injury and incurring more personal costs.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>General accomplishments: We worked on the design report and slides. Jessie looked into the AMD workflow for putting a model on the FPGA. We now have a more fleshed-out idea of the tasks that need to be done. More details can be found in Jessie\u2019s status report.\u00a0 Danny finished the Django tutorial.\u00a0 We asked Dueck\u2019s &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/2024\/09\/28\/team-status-report-for-09-28\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Team Status Report for 09\/28&#8221;<\/span><\/a><\/p>\n","protected":false},"author":936,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[13],"tags":[9,7],"_links":{"self":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/111"}],"collection":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/users\/936"}],"replies":[{"embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/comments?post=111"}],"version-history":[{"count":1,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/111\/revisions"}],"predecessor-version":[{"id":113,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/posts\/111\/revisions\/113"}],"wp:attachment":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/media?parent=111"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/categories?post=111"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/f24-teamc4\/wp-json\/wp\/v2\/tags?post=111"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}