{"id":123,"date":"2022-02-26T22:41:57","date_gmt":"2022-02-26T22:41:57","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/?p=123"},"modified":"2022-02-27T17:36:59","modified_gmt":"2022-02-27T17:36:59","slug":"charlies-status-report","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/2022\/02\/26\/charlies-status-report\/","title":{"rendered":"Charlie’s status report for 26 February 2022"},"content":{"rendered":"

\u201c<\/span>\u00a0What<\/span>\u00a0did<\/span>\u00a0you<\/span>\u00a0personally<\/span>\u00a0accomplish<\/span>\u00a0this<\/span>\u00a0week<\/span>\u00a0on<\/span>\u00a0the<\/span>\u00a0project?<\/span>\u00a0Give<\/span>\u00a0files<\/span>\u00a0or<\/span><\/strong>
photos<\/span>\u00a0that<\/span>\u00a0demonstrate<\/span>\u00a0your<\/span>\u00a0progress.<\/span>\u00a0Prove<\/span>\u00a0to<\/span>\u00a0the<\/span>\u00a0reader<\/span>\u00a0that<\/span>\u00a0you<\/span>\u00a0put<\/span>\u00a0sufficient\u00a0effort into the project over the course of the week (12+ hours).<\/span><\/strong><\/p>\n

This week, I implemented image segmentation using Detectron2 to identify the location of only humans in the scene.<\/p>\n

\"\"\"\"<\/p>\n

The image on the left is the test image provided by the Detectron2 dataset. Just to be certain that the image segmentation method works on non-full bodies and difficult images, I tested it on my own image. It appears that the Detectron2 works very well. Thereafter, I wrote the network in a modular method so that a function that outputs the location of each speaker can be easily called. The coordinates will then be combined with our angle estimation pipeline (which Larry is currently implementing).<\/p>\n

\u201c<\/span>\u00a0Is<\/span>\u00a0your<\/span>\u00a0progress<\/span>\u00a0on<\/span>\u00a0schedule<\/span>\u00a0or<\/span>\u00a0behind?<\/span>\u00a0If<\/span>\u00a0you<\/span>\u00a0are<\/span>\u00a0behind,<\/span>\u00a0what<\/span>\u00a0actions<\/span>\u00a0will<\/span>\u00a0be\u00a0<\/span>taken<\/span>\u00a0to<\/span>\u00a0catch<\/span>\u00a0up<\/span>\u00a0to<\/span>\u00a0the<\/span>\u00a0project<\/span>\u00a0schedule?<\/span><\/strong><\/p>\n

I would say we are slightly behind, because I was surprised at how difficult it was to separate the speech of two speakers even with the STFT method. I had previously implemented this method in a class and tested that it worked successfully. However, it did not work on our audio recordings. I am currently in the process of debugging why that occurred. In the meantime, Stella is working on delay-and-sum beamforming which is our second attempt at enhancing speech.<\/p>\n

\u201c<\/span>\u00a0What<\/span>\u00a0deliverables<\/span>\u00a0do<\/span>\u00a0you<\/span>\u00a0hope<\/span>\u00a0to<\/span>\u00a0complete<\/span>\u00a0in<\/span>\u00a0the<\/span>\u00a0next<\/span>\u00a0week?<\/span><\/strong><\/p>\n

In the coming week, I hope to be able to get the equations for delay-and-sum beamforming from stella. Once I receive the equations, I will then be able to implement the generalised sidelobe canceller\u00a0 (GSC) to determine if our speech enhancement method works (in a non-real-time case). In the event that GSC does not work, my group has identified a deep learning approach to separate speech. We do not plan to jump straight to that as it cannot be applied in realtime.<\/p>\n","protected":false},"excerpt":{"rendered":"

\u201c\u00a0What\u00a0did\u00a0you\u00a0personally\u00a0accomplish\u00a0this\u00a0week\u00a0on\u00a0the\u00a0project?\u00a0Give\u00a0files\u00a0orphotos\u00a0that\u00a0demonstrate\u00a0your\u00a0progress.\u00a0Prove\u00a0to\u00a0the\u00a0reader\u00a0that\u00a0you\u00a0put\u00a0sufficient\u00a0effort into the project over the course of the week (12+ hours). This week, I implemented image segmentation using Detectron2 to identify the location of only humans in the scene. The image on the left is the test image provided by the Detectron2 dataset. Just to be certain that the image segmentation method works on … <\/p>\n

Continue reading “Charlie’s status report for 26 February 2022”<\/span><\/a><\/p>\n","protected":false},"author":533,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[8],"tags":[9],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/posts\/123"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/users\/533"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/comments?post=123"}],"version-history":[{"count":2,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/posts\/123\/revisions"}],"predecessor-version":[{"id":127,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/posts\/123\/revisions\/127"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/media?parent=123"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/categories?post=123"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s22-teamd6\/wp-json\/wp\/v2\/tags?post=123"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}