{"id":49,"date":"2020-10-03T20:20:45","date_gmt":"2020-10-03T20:20:45","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/?p=49"},"modified":"2020-10-03T20:55:36","modified_gmt":"2020-10-03T20:55:36","slug":"echo-gaos-status-report","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/2020\/10\/03\/echo-gaos-status-report\/","title":{"rendered":"Echo Gao&#8217;s Status Update for 10\/3"},"content":{"rendered":"<h3><strong>Computer Vision:<\/strong><\/h3>\n<p>Compared different Computer Vision libraries (YOLO3, Tiny-YOLO, OpenCV, Tensorflow) and their detection speed &amp; accuracy on PC. Determined to use Tensorflow Lite for the highest frame rate.<\/p>\n<h6>1.YOLO3&amp; Tiny-YOLO<\/h6>\n<p>You only look once (<em>YOLO<\/em>) is a state-of-the-art, real-time object detection system.\u00a0 \u00a0Tiny-YOLO is a variation of the Yolo object detector. It is much faster than Yolo, yet less accurate.\u00a0For reference, Redmon et al. report ~51-57% mAP for YOLO on the COCO benchmark dataset while Tiny-YOLO is only 23.7% mAP \u2014 less than half of the accuracy of its bigger brothers (Rosebrock, Adrian).We used a pre-trained model from\u00a0https:\/\/pjreddie.com. Then we optimized tiny yolo by letting it only detects person and reduced input resolution from 416&#215;416 to 200&#215;200. However, it still only reaches 3.7 fps on MacOS. (We also tested out printing the mid-x position of the detected person, which is a value we will need for the robot to rotate and move accordingly).<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-large wp-image-36\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-2.45.30-PM-1024x403.png\" alt=\"\" width=\"840\" height=\"331\" srcset=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-2.45.30-PM-1024x403.png 1024w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-2.45.30-PM-300x118.png 300w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-2.45.30-PM-768x302.png 768w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-2.45.30-PM-1200x472.png 1200w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p>reference:<\/p>\n<p>https:\/\/pjreddie.com\/darknet\/yolo\/<\/p>\n<p>https:\/\/medium.com\/@madhawavidanapathirana\/real-time-human-detection-in-computer-vision-part-2-c7eda27115c6<\/p>\n<p>https:\/\/stackoverflow.com\/questions\/44674517\/yolo-darknet-detecting-only-specific-class-like-person-cat-dog-etc<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<h6>2.OpenCV<\/h6>\n<p>Haar Cascade Classifier is a popular algorithm for object detection. We used pre-trained Haar cascade models in OpenCV to detect human face in real time. It is the least accurate approach we&#8217;d inverstigated.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-large wp-image-37\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.32.33-PM-1024x693.png\" alt=\"\" width=\"840\" height=\"568\" srcset=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.32.33-PM-1024x693.png 1024w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.32.33-PM-300x203.png 300w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.32.33-PM-768x520.png 768w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.32.33-PM-1200x812.png 1200w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.32.33-PM.png 1424w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p>reference:<\/p>\n<p>https:\/\/opencv-python-tutroals.readthedocs.io\/en\/latest\/py_tutorials\/py_objdetect\/py_face_detection\/py_face_detection.html<\/p>\n<blockquote class=\"wp-embedded-content\" data-secret=\"3z3FGNCAOD\"><p><a href=\"https:\/\/techvidvan.com\/tutorials\/face-recognition-project-python-opencv\/\">Real-time Face Recognition with Python &#038; OpenCV<\/a><\/p><\/blockquote>\n<p><iframe class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" src=\"https:\/\/techvidvan.com\/tutorials\/face-recognition-project-python-opencv\/embed\/#?secret=3z3FGNCAOD\" data-secret=\"3z3FGNCAOD\" width=\"600\" height=\"338\" title=\"&#8220;Real-time Face Recognition with Python &#038; OpenCV&#8221; &#8212; TechVidvan\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe><\/p>\n<p>&nbsp;<\/p>\n<h6>3.Tensorflow<\/h6>\n<p>TensorFlow\u2122 is an open-source API from Google, which is widely used for solving machine learning tasks that involve Deep Neural Networks. We used Tensorflow Detection Model Zoo as our pre-trained model. It reached about 5-10 fps and its accuracy was good enough. Later, we found out that Tensorflow Lite is a even better optimized approach on raspberry pi.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-large wp-image-39\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.55.27-PM-1024x617.png\" alt=\"\" width=\"840\" height=\"506\" srcset=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.55.27-PM-1024x617.png 1024w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.55.27-PM-300x181.png 300w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.55.27-PM-768x463.png 768w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/Screen-Shot-2020-10-03-at-3.55.27-PM-1200x723.png 1200w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p>reference:<\/p>\n<p>https:\/\/medium.com\/@madhawavidanapathirana\/real-time-human-detection-in-computer-vision-part-2-c7eda27115c6<\/p>\n<p>https:\/\/github.com\/EdjeElectronics\/TensorFlow-Lite-Object-Detection-on-Android-and-Raspberry-Pi\/blob\/master\/Raspberry_Pi_Guide.md<\/p>\n<h3>Hardware:<\/h3>\n<p>Create2 robotic base, raspberry pi, NCS2 and the Pi camera were all received on Thursday, 30th Sep. We set up the Pi environment (installing the OS and relevant libraries) and run the CV algorithm on the Pi with low FPS.<\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-large wp-image-40\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1082-1024x768.jpg\" alt=\"\" width=\"840\" height=\"630\" srcset=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1082-1024x768.jpg 1024w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1082-300x225.jpg 300w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1082-768x576.jpg 768w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1082-1200x900.jpg 1200w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-large wp-image-41\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1081-1024x768.jpg\" alt=\"\" width=\"840\" height=\"630\" srcset=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1081-1024x768.jpg 1024w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1081-300x225.jpg 300w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1081-768x576.jpg 768w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-content\/uploads\/sites\/114\/2020\/10\/IMG_1081-1200x900.jpg 1200w\" sizes=\"(max-width: 709px) 85vw, (max-width: 909px) 67vw, (max-width: 1362px) 62vw, 840px\" \/><\/p>\n<p>Difficulties<\/p>\n<p>The Pi we borrowed from the ECE inventory is a Raspberry Pi 3 instead of Raspberry Pi 4, and there&#8217;s a huge performance difference between the two. Also, the NCS2 we chose as CV module is a relatively outdated piece of hardware, and is very hard to set up to coordinate with our choice of CV algorithm, Tensorflow Lite. After researching, in order to run the library, we need to use the NCS SDK and convert the module into NCS2 compatible, then run it on NCS2. On top of which, we found a result demo on YouTube and the result the video uploader was able to achieve on his Pi3 + NCS2 platform was 9-10 fps, which barely reaches the FPS requirement we had in mind.<\/p>\n<p>Intended Solutions<\/p>\n<p>We found another platform that outperforms the NCS2 and is specially optimized for running Tensorflow Lite. It&#8217;s called Google Coral, and it&#8217;s also a USB accelerator that can be connected to our Pi. The expected result of such hardware optimization will lead us to be running our CV algorithm at around 20 FPS. We also intend to get a refund for the NCS2 if possible.<\/p>\n<p>reference:<\/p>\n<blockquote class=\"wp-embedded-content\" data-secret=\"P9TQI8fqPQ\"><p><a href=\"https:\/\/www.pyimagesearch.com\/2019\/04\/08\/openvino-opencv-and-movidius-ncs-on-the-raspberry-pi\/\">OpenVINO, OpenCV, and Movidius NCS on the Raspberry Pi<\/a><\/p><\/blockquote>\n<p><iframe class=\"wp-embedded-content\" sandbox=\"allow-scripts\" security=\"restricted\" style=\"position: absolute; clip: rect(1px, 1px, 1px, 1px);\" src=\"https:\/\/www.pyimagesearch.com\/2019\/04\/08\/openvino-opencv-and-movidius-ncs-on-the-raspberry-pi\/embed\/#?secret=P9TQI8fqPQ\" data-secret=\"P9TQI8fqPQ\" width=\"600\" height=\"338\" title=\"&#8220;OpenVINO, OpenCV, and Movidius NCS on the Raspberry Pi&#8221; &#8212; PyImageSearch\" frameborder=\"0\" marginwidth=\"0\" marginheight=\"0\" scrolling=\"no\"><\/iframe><\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Computer Vision: Compared different Computer Vision libraries (YOLO3, Tiny-YOLO, OpenCV, Tensorflow) and their detection speed &amp; accuracy on PC. Determined to use Tensorflow Lite for the highest frame rate. 1.YOLO3&amp; Tiny-YOLO You only look once (YOLO) is a state-of-the-art, real-time object detection system.\u00a0 \u00a0Tiny-YOLO is a variation of the Yolo object detector. It is much &hellip; <a href=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/2020\/10\/03\/echo-gaos-status-report\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Echo Gao&#8217;s Status Update for 10\/3&#8221;<\/span><\/a><\/p>\n","protected":false},"author":347,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3,5],"tags":[9,10],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/posts\/49"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/users\/347"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/comments?post=49"}],"version-history":[{"count":6,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/posts\/49\/revisions"}],"predecessor-version":[{"id":51,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/posts\/49\/revisions\/51"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/media?parent=49"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/categories?post=49"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f20-teamb1\/wp-json\/wp\/v2\/tags?post=49"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}