{"id":170,"date":"2021-04-10T18:33:54","date_gmt":"2021-04-10T18:33:54","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/?p=170"},"modified":"2021-04-10T18:33:54","modified_gmt":"2021-04-10T18:33:54","slug":"chriss-status-report-for-apr-10","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/2021\/04\/10\/chriss-status-report-for-apr-10\/","title":{"rendered":"Chris\u2019s Status Report for Apr 10"},"content":{"rendered":"<p><span style=\"font-weight: 400\">\u2019ve been working on 2 things this week. The first thing I\u2019ve been working on is trying to find how our neural networks perform on different types of combinations of tasks. Since our current game requires the discrimination between 2 types of action, we collected data from 8 different actions and compared the performance of each neural network on every possible 2-pair of actions. We\u2019ve found that similar actions are hard to be distinguished, and very different actions are easy to be distinguished. For example, thinking about lifting the right arm and left arm are hard to distinguish, but not thinking about anything and moving the eyeball from right to left is easy to distinguish. We found that the DeepConvNet performs the best overall, so here we will only include the results from this network.<\/span><\/p>\n<p><img decoding=\"async\" loading=\"lazy\" class=\"alignnone size-medium wp-image-171\" src=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-content\/uploads\/sites\/145\/2021\/04\/WX20210410-143242@2x-300x261.png\" alt=\"\" width=\"300\" height=\"261\" srcset=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-content\/uploads\/sites\/145\/2021\/04\/WX20210410-143242@2x-300x261.png 300w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-content\/uploads\/sites\/145\/2021\/04\/WX20210410-143242@2x-768x668.png 768w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-content\/uploads\/sites\/145\/2021\/04\/WX20210410-143242@2x-1024x891.png 1024w, http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-content\/uploads\/sites\/145\/2021\/04\/WX20210410-143242@2x.png 1078w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/p>\n<p><span style=\"font-weight: 400\">The actions from are numbered from 0-7, and the color of the matrix shows how good the performance of distinguishing the two actions are. From 0-7 the actions are respectively:<\/span><\/p>\n<ul>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;smile&#8217;,<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;stare&#8217;,<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;open-eyes-neutral&#8217;,<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;closed-eyes-neutral&#8217;,<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;left-to-right-eyeball&#8217;,<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;right-to-left-eyeball&#8217;,<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;think-lifting-left-arm&#8217;,<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">\u00a0\u00a0\u00a0\u00a0&#8216;think-lifting-right-arm&#8217;<\/span><\/li>\n<\/ul>\n<p><span style=\"font-weight: 400\">And as evident in the plot, the action pairs that are most easily distinguishable are:<\/span><\/p>\n<ol>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Eyeball moving right to left and think lifting left arm<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Closed eye neutral and left to right eyeball<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Open eyes neutral and closed eyes neutral<\/span><\/li>\n<li style=\"font-weight: 400\"><span style=\"font-weight: 400\">Smile and closed eyes neutral<\/span><\/li>\n<\/ol>\n<p><span style=\"font-weight: 400\">As we can see from these pairs, the algorithm almost has to rely on some eyeball information to operate. This could be due to multiple factors, one of them being we are not cleaning the data at present stage. The data cleaning pipeline is still under progress, and all the training and testing here are done on raw signals. We will continue testing and keep a note on how these performances change. However, what\u2019s worth noting is that even if we were to rely on eyeball movement signals for our game\u2019s control signal, the technology we are using is still significantly different from just using eye tracking.<\/span><\/p>\n<p>&nbsp;<\/p>\n<p><span style=\"font-weight: 400\">The second thing I\u2019ve been doing is adding multi thread processing into the main game logic. I take the main files Eryn has been working on and I\u2019m writing a multi-process consumer and producer system that reads in signals and processes signals, then send the control signals to the game logic.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u2019ve been working on 2 things this week. The first thing I\u2019ve been working on is trying to find how our neural networks perform on different types of combinations of tasks. Since our current game requires the discrimination between 2 types of action, we collected data from 8 different actions and compared the performance of &hellip; <\/p>\n<p class=\"link-more\"><a href=\"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/2021\/04\/10\/chriss-status-report-for-apr-10\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Chris\u2019s Status Report for Apr 10&#8221;<\/span><\/a><\/p>\n","protected":false},"author":447,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[5],"tags":[],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/posts\/170"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/users\/447"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/comments?post=170"}],"version-history":[{"count":1,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/posts\/170\/revisions"}],"predecessor-version":[{"id":172,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/posts\/170\/revisions\/172"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/media?parent=170"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/categories?post=170"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s21-teamd4\/wp-json\/wp\/v2\/tags?post=170"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}