Accomplishments
This week I’ve been working on the FPU Job Manager Operations. I’ve been following my previous process of describing the FSM control signals state-by-state, then simply copying them over into SystemVerilog. Here’s what I have so far:
| Operation | Described? | Implemented? | Testbench? |
| Linear Forward | Yes | Yes | Yes |
| Linear Backward | Yes | Yes | |
| Linear Weight Gradient | Yes | Yes | |
| Linear Bias Gradient | Yes | Yes | |
| Convolution Forward | Yes | ||
| Convolution Backward | Yes | Yes | |
| Convolution Weight Gradient | Yes | ||
| Convolution Bias Gradient | Yes | ||
| MaxPool Forward | Yes | ||
| MaxPool Backward | |||
| ReLU Forward | Yes | ||
| ReLU Backward | Yes | ||
| Softmax Forward | |||
| Softmax Backward | |||
| Cross-Entropy Backward | |||
| Flatten Forward | Yes | ||
| Flatten Backward | Yes | Yes | |
| Parameter Update | Yes |
Last week, I had the Convolutional Forward and Linear Forward operations described, and only the Linear Forward operation implemented.
I’ve consolidated all of the weight and bias operations into a single “Parameter Update” operation, since they’re all the exact same and the shape of each tensor can be read from memory.
Another work-around I’m implementing is for the Softmax Backward operation. I haven’t been able to find a working floating-point exponent calculator in Verilog, so in the case that I’m unable to find one, I will simply subtract the output from the label, which in terms of optimization will have the same effect as taking the backwards gradient of the softmax direction.
Schedule & Accomplishments for Next Weeks
I’ll be finishing up the FPU Job Manager operations over the next couple days, then preparing the Model Manager for the Demo.