{"id":169,"date":"2022-11-12T23:05:43","date_gmt":"2022-11-12T23:05:43","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/?p=169"},"modified":"2022-11-12T23:05:43","modified_gmt":"2022-11-12T23:05:43","slug":"team-status-report-for-11-12-2022","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/2022\/11\/12\/team-status-report-for-11-12-2022\/","title":{"rendered":"Team Status Report for 11\/12\/2022"},"content":{"rendered":"<ol>\n<li style=\"font-weight: 400\"><b>What are the most significant risks that could jeopardize the success of the project?<\/b><\/li>\n<\/ol>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Pre-processing:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Currently, I am relying on openCV\u2019s \u201ccv.2connectedComponentsWithStats()\u201d function that outputs various statistical values in regards to the original image inputted, including the left, top coordinates as well as the width, height, and area of the most commonly appearing object (braille dots in our case). However, depending on the lighting or the image quality of the original image taken, the accuracy of this stats function needs to be further tested in order to modify further modifications.\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Classification:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">On the classification side, one new risk that was introduced when testing our neural network inference on the Jetson Nano was latency. Since each character has around a 0.1s latency, if we were to process characters sequentially, an especially long sentence could produce substantial latency.<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Hardware:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">The Jetson Nano hardware also presented some challenges due to its limited support as a legacy platform in the Jetson ecosystem. Missing drivers and slow package build times make bring-up particularly slow. This is, however, a one-time cost which should not have any significant impact on our final product.<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Post-processing:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Another hardware related risk to our final product is the audio integration capabilities of the Nano. Since this is one of the last parts of integration, complications could be critical.\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><b>2. How are these risks being managed?\u00a0<\/b><\/p>\n<p>&nbsp;<\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Pre-processing:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">On primary level, pixel by pixel comparison between image and printed matrices on terminal would be undergone to understand the current accuracy level and for further tweaking of the parameters. Furthermore, cv\u2019s non_max_suppression() function is being further investigated to mitigate some of the inaccuracies that can rise from the initial \u201cconnectedComponentsWithStats().\u201d\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Classification:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">To address possible latency issues as a result of individual character inference latency, we are hoping to convert our model from the mxnet framework to NVIDIA\u2019s TensorRT, which the Jetson can use to run the model on a batch of images in parallel. This should reduce the sequential bottleneck that we are currently facing.<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Hardware:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Since hardware risks are a one-time cost, as mentioned above, we do not feel that we will need to take steps to manage them at this time. However, we are considering using a docker image to cross-compile larger packages for the Jetson on a more powerful system.<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Post-processing:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">After finishing camera integration, we will work on interacting with audio through the usb port. We have a stereo adapter ready to connect to headphones.<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><b>3. What contingency plans are ready?<\/b><span style=\"font-weight: 400\">\u00a0<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Classification:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">If the inference time on the Jetson Nano is not significantly improved by moving to TensorRT, one contingency plan we have in place would be to migrate back to the Jetson AGX Xavier, which has significantly more computing power. While this comes at the expense of portability and power efficiency, it is within the parameters of our original design<\/span><\/li>\n<\/ul>\n<\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Post-Processing:<\/span>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">There is a possible sound board input and output pcb that would allow us to attach to the nano and play sound. This comes with added expense and complexity, but it seems more likely to be proven effective.\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><b>4. Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)?<\/b><\/p>\n<p><span style=\"font-weight: 400\">Integrating each of our individual components into our overall software pipeline did not introduce any obvious challenges. Therefore, we did not think it is necessary to make any significant changes to our software system. However, in response to interim demo feedback, we are looking to create more definitive testing metrics when deciding on certain algorithms or courses of action. This will allow us to justify our choices moving forward and give our final report clarity. In addition to the testing, we are considering a more unified interaction between classification and post-processing that helps create a more deterministic approach to which characters might be wrong more often.\u00a0<\/span><\/p>\n<p><b>5. Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?<\/b><span style=\"font-weight: 400\">\u00a0<\/span><\/p>\n<p><span style=\"font-weight: 400\">The minor changes that we are making to the individual subsystems are crucial for the efficiency and effectiveness of our product. Making sure that we stay on top of optimal decisions and advice given by our professors and TAs.\u00a0<\/span><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What are the most significant risks that could jeopardize the success of the project? Pre-processing: Currently, I am relying on openCV\u2019s \u201ccv.2connectedComponentsWithStats()\u201d function that outputs various statistical values in regards to the original image inputted, including the left, top coordinates as well as the width, height, and area of the most commonly appearing object (braille &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/2022\/11\/12\/team-status-report-for-11-12-2022\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Team Status Report for 11\/12\/2022&#8221;<\/span><\/a><\/p>\n","protected":false},"author":646,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[5],"tags":[9],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/posts\/169"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/users\/646"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/comments?post=169"}],"version-history":[{"count":1,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/posts\/169\/revisions"}],"predecessor-version":[{"id":170,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/posts\/169\/revisions\/170"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/media?parent=169"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/categories?post=169"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f22-teamb1\/wp-json\/wp\/v2\/tags?post=169"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}