{"id":181,"date":"2020-04-12T17:15:35","date_gmt":"2020-04-12T17:15:35","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/?p=181"},"modified":"2020-04-12T17:15:35","modified_gmt":"2020-04-12T17:15:35","slug":"theodors-status-report-for-april-11","status":"publish","type":"post","link":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/2020\/04\/12\/theodors-status-report-for-april-11\/","title":{"rendered":"Theodor&#8217;s Status Report for April 11"},"content":{"rendered":"<p><strong>Accomplishments<\/strong><\/p>\n<p>This week I&#8217;ve been working on the FPU Job Manager Operations. I&#8217;ve been following my previous process of describing the FSM control signals state-by-state, then simply copying them over into SystemVerilog. Here&#8217;s what I have so far:<\/p>\n<table dir=\"ltr\" border=\"1\" cellspacing=\"0\" cellpadding=\"0\">\n<colgroup>\n<col width=\"189\" \/>\n<col width=\"100\" \/>\n<col width=\"100\" \/>\n<col width=\"100\" \/><\/colgroup>\n<tbody>\n<tr>\n<td>Operation<\/td>\n<td>Described?<\/td>\n<td>Implemented?<\/td>\n<td>Testbench?<\/td>\n<\/tr>\n<tr>\n<td>Linear Forward<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<\/tr>\n<tr>\n<td>Linear Backward<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Linear Weight Gradient<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Linear Bias Gradient<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Convolution Forward<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Convolution Backward<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Convolution Weight Gradient<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Convolution Bias Gradient<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>MaxPool Forward<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>MaxPool Backward<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>ReLU Forward<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>ReLU Backward<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Softmax Forward<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Softmax Backward<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Cross-Entropy Backward<\/td>\n<td><\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Flatten Forward<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Flatten Backward<\/td>\n<td>Yes<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<\/tr>\n<tr>\n<td>Parameter Update<\/td>\n<td>Yes<\/td>\n<td><\/td>\n<td><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Last week, I had the Convolutional Forward and Linear Forward operations described, and only the Linear Forward operation implemented.<\/p>\n<p>I&#8217;ve consolidated all of the weight and bias operations into a single &#8220;Parameter Update&#8221; operation, since they&#8217;re all the exact same and the shape of each tensor can be read from memory.<\/p>\n<p>Another work-around I&#8217;m implementing is for the Softmax Backward operation. I haven&#8217;t been able to find a working floating-point exponent calculator in Verilog, so in the case that I&#8217;m unable to find one, I will simply subtract the output from the label, which in terms of optimization will have the same effect as taking the backwards gradient of the softmax direction.<\/p>\n<p><strong>Schedule &amp; Accomplishments for Next Weeks<\/strong><\/p>\n<p>I&#8217;ll be finishing up the FPU Job Manager operations over the next couple days, then preparing the Model Manager for the Demo.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Accomplishments This week I&#8217;ve been working on the FPU Job Manager Operations. I&#8217;ve been following my previous process of describing the FSM control signals state-by-state, then simply copying them over into SystemVerilog. Here&#8217;s what I have so far: Operation Described? Implemented? Testbench? Linear Forward Yes Yes Yes Linear Backward Yes Yes Linear Weight Gradient Yes &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/2020\/04\/12\/theodors-status-report-for-april-11\/\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;Theodor&#8217;s Status Report for April 11&#8221;<\/span><\/a><\/p>\n","protected":false},"author":246,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[3],"tags":[],"_links":{"self":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/posts\/181"}],"collection":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/users\/246"}],"replies":[{"embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/comments?post=181"}],"version-history":[{"count":1,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/posts\/181\/revisions"}],"predecessor-version":[{"id":182,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/posts\/181\/revisions\/182"}],"wp:attachment":[{"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/media?parent=181"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/categories?post=181"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/course.ece.cmu.edu\/~ece500\/projects\/s20-teama7\/wp-json\/wp\/v2\/tags?post=181"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}