Accomplishments
This week I’ve been working on the FPU Job Manager Operations. I’ve been following my previous process of describing the FSM control signals statebystate, then simply copying them over into SystemVerilog. Here’s what I have so far:
Operation 
Described? 
Implemented? 
Testbench? 
Linear Forward 
Yes 
Yes 
Yes 
Linear Backward 
Yes 
Yes 

Linear Weight Gradient 
Yes 
Yes 

Linear Bias Gradient 
Yes 
Yes 

Convolution Forward 
Yes 


Convolution Backward 
Yes 
Yes 

Convolution Weight Gradient 
Yes 


Convolution Bias Gradient 
Yes 


MaxPool Forward 
Yes 


MaxPool Backward 



ReLU Forward 
Yes 


ReLU Backward 
Yes 


Softmax Forward 



Softmax Backward 



CrossEntropy Backward 



Flatten Forward 
Yes 


Flatten Backward 
Yes 
Yes 

Parameter Update 
Yes 


Last week, I had the Convolutional Forward and Linear Forward operations described, and only the Linear Forward operation implemented.
I’ve consolidated all of the weight and bias operations into a single “Parameter Update” operation, since they’re all the exact same and the shape of each tensor can be read from memory.
Another workaround I’m implementing is for the Softmax Backward operation. I haven’t been able to find a working floatingpoint exponent calculator in Verilog, so in the case that I’m unable to find one, I will simply subtract the output from the label, which in terms of optimization will have the same effect as taking the backwards gradient of the softmax direction.
Schedule & Accomplishments for Next Weeks
I’ll be finishing up the FPU Job Manager operations over the next couple days, then preparing the Model Manager for the Demo.