I’ve got the CONV 3×3 implementation working with INT8, which is a solid milestone for the hardware pipeline and gets us closer to the final architecture we need for real-time inference. Right now I’m focusing on making the whole thing parametizable so we can easily adjust configurations without having to rewrite a bunch of code every time we want to change kernel sizes, data widths, or number of channels. It’s coming along pretty well and the basic structure is there, but there’s still some refactoring to do to make it clean and flexible enough to handle different use cases down the line. The goal is to have something that’s modular enough that we can swap in different layer configurations without breaking everything, which will be really helpful once we start optimizing for different parts of the YOLO network.
On the model side, I’ve been spending most of my time training with traffic signs datasets to get the detector tuned properly for our specific application and use case. The goal is to get good accuracy on sign detection specifically since that’s what we’re targeting for the real-time demo on the Kria board, and we want to make sure it actually works well in real-world conditions.
I’m also deep in the research phase for quantization strategies, which is pretty critical for getting this running efficiently on the FPGA without sacrificing too much accuracy. Trying to figure out the best approach to get from floating point weights and activations down to INT8 without totally tanking the accuracy or introducing too much quantization error that would mess up our detection performance. There are a few different techniques I’m looking into – post-training quantization which is simpler but might lose more accuracy, quantization-aware training which should preserve accuracy better but takes more time, and some hybrid approaches that try to get the best of both worlds. I’m still working through the tradeoffs and running experiments to figure out what makes the most sense for our FPGA implementation and performance requirements, especially considering we need to hit that sub-30ms inference latency target while maintaining reasonable detection accuracy on traffic signs.