Posted on

Joshua’s Status Update for 9/18/21

This week, I worked on refining the details on our implementation with my team. I looked into possible algorithms that could be used on our project with my team, and also furthered my knowledge of upscaling algorithms that were used in the past, specifically on browsers, since our initial use case was video/movie upscaling. After changing our use case, I looked at groups from previous semesters who had similar projects to ours, and compared their numbers to our estimates for throughput and image/video processing on FPGAs. I also looked into the process of down-scaling videos from a higher quality, as this will be an important part of the testing for our project.

Since I fell ill on Wednesday and did not recover for 3 days, I did not meet with my team on that day, and did not spend as much time on research as I had initially planned. However, we still managed to achieve most of our initial goals for the week, and my team and I have communicated and managed to bring me up to speed with our progress into the research of our project.

I also helped write the introduction of the project and setup the tabs needed for our blog/website, as well as compiling the progress of my team to write the team status report.

Posted on

Team Status Report for 9/18/21

This week, my group met with our TA, Joel, and Bryon to refine our abstract and pinpoint the finer details of our implementation. Taking on their recommendations, we decided to change our use case, specifically, to security/video streaming. Allowing to up-scale videos on demand in real-time allows for greater security and better decision-making when the user is presented with potential threats. Also, client-side upscaling of videos can make up for poor internet connection. After considering the throughput and use case, we also decided to go with 24fps as a target instead of 60fps, as this is more realistic whilst still perceivable as a video to the user.

As Byron suggested, we examined existing upscaling methods used in browsers more in-depth, as well as reading up on some DSP literature that he sent us. We decided that neural net methods were indeed more useful for our implementation, and we are in the process of figuring out how to fit this architecture onto an FPGA.

For the coming week, we will further develop our schedule, and also confirm how we will procure the key components of our project. We will also setup team infrastructure such as Github to ensure we can coordinate our progress better. Overall, we are on schedule and ready to progress to the next week.

Posted on

Kunal’s Status Update

This week our team worked on pinpointing an algorithm for usage in the real-time video upscaling problem. We found that DSP algorithms approach the problem in a rather naive way, as they’re unable to scale out to different form factors for the video data. The inputs to the image upscaling problem are uniformly distributed but often vary in slight ways on each iteration, and hence a deep learning based approach is favored. 

 

The deep learning algorithm I looked into was image super-resolution from sparsity. This algorithm covered how we can take batches of pixels from a low resolution image and build out 2 matrices representing a downsampling & blurring filter. The deep learning algorithm would be based on a classical layered neural network taking in pixel densities and locations as inputs. This algorithm will then train two dictionaries both representing a sparse coding for the image upscaling algorithm. Two dictionaries for both the low resolution and super-resolution images would then be correlated and through the iterative process of gradient descent we can figure the appropriate heuristics for the trained model. 

Posted on

James’s Status for 9/18

This week I did research on other image upscaling methods in recent literature as well as in classical DSP modes, honed in on an algorithm to pursue & reasoning behind it. The DSP ways seem to be lossy with respect to high frequency information or much more involved with respect to mathematical processing that would parallel computation going into convolutional net ways. Most modern upscaling being researched currently uses neural net methods to upscale images, and we are going to go down this route as well.

I also did work on the outline of the presentation; there are still a few parts that need to be finished up shortly. Tomorrow’s plan is to get the presentation done and ready for the presentations this week.

Lastly, I also got this blog up with our project name and member names.