This week our team worked on pinpointing an algorithm for usage in the real-time video upscaling problem. We found that DSP algorithms approach the problem in a rather naive way, as they’re unable to scale out to different form factors for the video data. The inputs to the image upscaling problem are uniformly distributed but often vary in slight ways on each iteration, and hence a deep learning based approach is favored.
The deep learning algorithm I looked into was image super-resolution from sparsity. This algorithm covered how we can take batches of pixels from a low resolution image and build out 2 matrices representing a downsampling & blurring filter. The deep learning algorithm would be based on a classical layered neural network taking in pixel densities and locations as inputs. This algorithm will then train two dictionaries both representing a sparse coding for the image upscaling algorithm. Two dictionaries for both the low resolution and super-resolution images would then be correlated and through the iterative process of gradient descent we can figure the appropriate heuristics for the trained model.