Kunal’s Status Report (10/30)

This week I (Kunal) along with James worked on the I/O portion of the project. I’m currently building a mechanism to take mp4 data and break it down into frames for the upscaling algorithm implemented & placed on the fpga. Fundamentally, our algorithm is dependent on the streaming & deconstruction of video from which the proper mechanisms for upscaling are then ported onto the fpga unit. I’m currently implementing this through OpenCV and it’s libraries in C++. The host program that will communicate with the fpga I’m also writing in C++ & will be the core of how the real-time streaming aspect of this system will work. The implementation involves pinning devices to a registry, where in our case we only have 1 external device and that’s our CNN implementation on the hardware device. This is an iterative process that involves checking the programmability of a peripheral device and if the device is programmable then I break out of the loop and use that context to send kernels of data to the fpga unit. The speed at which the decoding of the mp4 into frames and then the rate at which the frames hit the fpga is an important heuristic which I’m working on optimizing for real-time streaming. Ideally we’d want to maintain the fps of the video as it is streamed into the fpga unit. For this to happen, the latencies of decoding the mp4 is going to be crucial and we will be benchmarking this part of the real-time streaming pipeline extensively.