Month: April 2022

This week I worked primarily on optimizing the fluid simulation algorithm on the fabric. This involved iterating upon the algorithm and exploring different ways to restructure the hardware and taking advantage of the HLS pragmas to allow the algorithm to run faster.

This also required figuring out what optimizations would break the implementation, and was a fairly iterative process. I was also working on developing the slides for our final presentation this week.

I think that we are on track in our schedule. Next week I will continue work on optimizing, and also work on our poster and video.

This week, we got a lot of integration work done.

 

On the software side, Alice finished up implementing the algorithm to support a hardware-friendly compilation. After verifying the results, we determined that it would be sufficient in terms of rendering a product with an appreciable simulation quality. We did have a small hiccup were the synthesis resulted in using 1500+ BRAMs (far more than the 460 we have on the board); however, after rebalancing some constants, we were able to fit everyhting in the device footprint.

 

On the hardware side, Ziyi finished up accelerating the data transfer interface bewteen the FPGA fabric and the FPGA host CPU and began investigating potential improvements for step5 of the kernel, and Jeremy began investigating some optimizations for unrolling and pipelining step2 and step3 in the kernel as well as inlining different function calls in order to reduce the latency of certain instructions.

As per our goals next week, we want to finish up accelerating and benchmarking our different improvements to the kernel. Once we have some appreciable results, we will begin assembling all of our presentation materials.

 

This was another good week for progressing in hardware-land. The first major contribution of the week expanding the AXI port so that we could transfer a whole Vec3 per transfer, rather than just a single position primitive (our 24 bit particle_pos_t datatype). The simpel effect of this optimization is that if we can move more data per transaction, this means we need fewer transactions to move all the data and thus spend much less time. In the simple example of grouping the three primitives together, this means that we’ll have three times fewer transactions overall, which roughly corresponds to a three times speedup. If we wanted to further send multiple Vec3s per transaction, we could save even more time; however, this could also lead to us hitting the upper bound of a 4kB page per burst transfer.

In order to implement the port widening, we needed to create an arbitrarily-sized datatype that is 3 times the width of a single primitive. Then, we would cast our writes to the output port to the 3-wide packed datatype. This seemed to make vitis happy enough to pack the data together.

Related to port widening, the next major contribution was implementing pipelined burst AXI4 transfers. Basically, the point of having a pipelined AXI transfer is that you amortize away the setup costs of having an isolated transfer and you gain significant throughput boosts from having a pipelined transfer.

However, it should be noted that in order to widen the ports, we needed to preprocess the particles position array by transfering every data value into a contiguous BRAM. This constitutes a pretty obvious design tradeoff for our project, where we expend more resources (and time!) in the effort of saving even more time overall.

 

As for next week, my next task is to accelerate the step5 loop and finish verifying the data transfer interface.

This week was another week of great progress! The first thing I did this week was to resolve the interfacing issues with the FPGA. After debugging the segfault with a dummy kernel, I was able to get it to successfully transmission from the FPGA fabric to the host CPU. From here, I just switched out the dummy kernel for the most up to date kernel (more on this later) and uploaded the full project onto the board, and presto! Results!

The next thing that I helped with was synthesizing the most up to date kernel (as mention above). This kernel was a major milestone in that it was the first implementation to include everything (including the nearest neighbor algorithms). While Alice and Jeremy mostly handled the algorithmic part of the implementation, I handled some of the Vitis build errors. One example of which was a dependency between different iterations of the writeback loop. After analyzing the loop body, I was able to fix this bug by introducing the dependency pragma, which allowed Vitis HLS to correctly optimize this.

As an aside, solving the different HLS build warnings is incredibly important. As programmers, the traditional “wisdom” is that warnings are more or less ignorable. The issue with HLS is that in order to adapt to the warning, Vitis HLS will expend a lot of extra unneccessary hardware resources, potentially an order of magnitude more than what the board supports!

My primary task next week is to investigate and document different optimizations and pragmas we can use to accelerate the performance of the kernel. Another tasks is to potentially investingating refactoring the code so that creating the particles also happens off-chip. This would free up some extra space for extra unrolling and optimizations.

Last week my goal was to work on some optimizations for the build. However, I had to shift my attention to what Jeremy was working on with the nearest neighbors. Since hardware needs to have known memory sizes for everything, it makes it challenging to discretize the 3D world space and partition the fluid particles accordingly, so we are currently trying to bound the possible fluid simulation positions. However, right now we still have a bit of work to do with re-working how we read particles in (in case they get dropped) and making sure the output is reasonable. My goal for next week is to finish up getting the algorithm to work correctly in hardware and then work on optimizations.

I definitely feel slightly behind since our team was not able to meet in person at all this week and communication has been slow. However, it seems like Jeremy will be able to meet in person again soon and we’ll be able to get back on track.

So this week unfortunately I have been sick with COVID, and wasn’t able to make as much progress as I wanted to originally. For the majority of the week I have been trying to get rest and recover quickly, so I’ll be productive next week. Later in the week I was able to put in some work where I worked on the correctness of our algorithm, as we were facing some issues where particles did not behave as we expected them to. The issues turned out to be with how we calculate the bounds of the scene and the voxels that particles are in, but they are mostly solved now.

I feel slightly behind since I did not expect to get sick this week, but I think that we are still in a good spot, and will be able to produce good results from our project. Next week I am ti help with collecting data and continuing to optimize our implementation.

Last week we were rushing to finish the build for the interim demo, and I was unable to complete the evaluation script. This week I was able to do so. I also read up a lot on unrolling, pipelining, and other optimizations outlined here: https://docs.xilinx.com/r/en-US/ug1399-vitis-hls/HLS-Pragmas

This upcoming week I’ll be working on adding said pragmas for optimizations. I’ll also be helping to verify that the Vitis HLS project provides reasonable fluid simulation output once Ziyi finishes the interface work to get an output text file from the FPGA.

We had a really great push at the beginning of this week. I’m optimistic that we can achieve our goals.

This week we got a ton done on the hardware side of things. First of all, after finally squeezing out the final few compilation errors, we managed to get a trimmed down version of the kernel (everything sans nearest neighbor loops) built using Vitis HLS synthesis. After looking at the synthesis analysis, we were able to see that the correct loops were also getting synthesized, which meant that our code was being inferred correctly.

After passing synthesis, we were able to export the rtl implementation (the .xo file) and import it into regular Vitis itself without any additional troubles. Then, after implementing some basic interfacing code between the host program and the kernel, were were able to build a binary to run on the FPGA. In order to get this binary onto the board, we simply scp’d the files onto the board.

Running the program resulted in a segmentation fault, so we still have a bit to go. My goal for next week is to debug the interface and work on some optimizations with Alice and Jeremy.