Team Status Report
UPDATES
We have a working end-to-end demo! It is currently very slow, unfortunately. We are working on improving the performance of our system, but we have a video processed on our system to demonstrate. Here is the original video and the seam-carved video.
RISKS
Now that we have a working demo, many of our risks have been mitigated. The greatest risk is that we won’t have the time to achieve the speedup that we want. However, even if we can’t, we still have a viable demo.
DESIGN CHANGES
We tested a new design for our energy function, which is a simple model of the “forward energy” used in the paper on video seam carving. In this new function, the energy of a pixel at <x, y, t> is the difference between the pixels at <x, y, t+1> and <x, y, t–1>. Removing the pixel with the lowest energy is then roughly equivalent to removing the pixel that leaves the smoothest output. This change improved our output quality a lot, as can be seen in our video above.
Maxwell Johnson
Personal work and Progress:
This week, I spent many hours in the lab getting our demo ready. John and I finished the Ethernet interface between the SoC and the external computer. I ported all of Riki’s code to run on the SoC, since it originally used many OpenCV structures that we couldn’t use on the standalone program we’re running. After everything was finally running together, we still had a couple bugs, most of which I ironed out in the last couple days. Between all that work, I didn’t have much time to work on the DMA solution until yesterday. Currently, the main bottleneck in the system is the energy computation, which is slow because all the data has to flow through the BRAM one 4-byte word at a time. It is also slow because while this transfer happens, the SoC can’t do any other work. DMA uses 1024-bit words, which will make the transfer itself much faster. It is also controlled by hardware outside of the SoC, so the processor can work on finding seams while the energies are being computed. I’m hoping to have the DMA solution working early next week.
Deliverables:
Speedup. I’ve made good progress on the DMA solution, which I’ll complete this week. With our other deliverables due this week, that may be the last optimization that I have time to make.
John Zhang
Personal Work
I got the Ethernet interface between the SoC and PC working! Now we have an end-to-end demo!
Progress
This week’s work has been mainly focused on achieving data transfer between the PC and the SoC. Last week I had an TCP echo server up on the SoC board, but wasn’t able to transfer data. This week I started by writing a client script in python using socket programming to try and send data to the server on the board. I wasn’t sure it would work because the server is a light-weight IP RAW app, not using the lwip socket API. But it turned out that the communication worked out just fine. I was able to send data to the board and receive data from it.
After that, I started trying my hand at programming the server so that it writes the data received to the DRAM. At first I was intending to hardcoding which memory address to write the video data, but there was a lot of segfault causing the server to hang. But then the good-ol’ malloc came to the rescue.
I also created a nice template for our team’s final presentation
Deliverables
In the next week I aim to improve the timing on pre-processing and post-processing of the video. We put together a python script last minute for the in-lab demo, there’s still lots of room for optimization.
Riki Khorana
Personal Work and Progress
This week was a hectic one for me, as I was busy preparing and tearing down for one of the biggest events we have for my student organization. I was given the task of mounting the C++ code written by me onto the SoC, however could not fulfill the task to a satisfactory level and had to ask Maxwell to help me out. I prototyped the pre and post processing applications with Python, and asked John to help me refine the code. In the end, I am entirely done with the software algorithm, and now onto finding some videos that the algorithm may work well with.
Deliverables
The final presentation is coming up, which I will have to work on. I will also be looking for better videos that fit our algorithm, so we have a good deliverable to show during the presentation.