status report – Page 4 – Team C4: HoloPyramid

Team Status Report for 3-6-21

This week, our team worked on our design review presentation and decided on the identity of our important hardware, and then we subsequently ordered OV7670 cameras and a DE2-115 board, which should arrive early next week.

We decided on the OV7670 cameras for the reasons mentioned previously in Jullia’s last status report: it is the analog camera that supports our requirements (analog output) and also has the most supporting literature, as it is a common camera used with FPGA. These cameras are also relatively inexpensive and only used up <$30 of our budget. We also decided on the DE2-115 (Cyclone IV) board, over other FPGA alternatives such as the DE0-CV (Cyclone V), for the following reasons: The DE2-115 board has a large number of logic elements and enough SDRAM to support storing a frame of image data as well as the necessary modules to interface with cameras and FPGA. The DE2-115 board has a sufficient number of GPIO pins to support four OV7670 cameras, with the use of a daughter board. This board is also within the ECE department’s parts inventory and thus easy (and free) to acquire and use, for the duration of our Capstone project.

This aforementioned hardware represents our current, most significant risk: successfully interfacing with four OV7670s cameras with the FPGA board. We need to link together the cameras’ control and data lines with the FPGA, which could be 72 pins at maximum. Some of these connections should be able to be linked together or otherwise not connected to GPIO pins, such as power and ground. However, this is a large enough number of needed GPIO connections that requires us to most likely need to buy a daughter board in order to have additional GPIO connections on the DE2-115 board. We have a couple risk mitigation plans if the daughter board does not work: One plan is to ignore a data line from each camera, the LSB of color data, in order to manage the connections. At absolutely worst, we can also create a holographic pyramid with less than four cameras, projecting fewer than four side-views of the object onto the pyramid.

We have also updated our schedule in order to prioritize the most critical components of our project, pushing back the implementation of image filters in favor of implementing the pyramid and live studio earlier. This schedule is shown below:

Jullia Tran’s Status Report for 3-6-21

This week, I work on preparing the Design presentation. I researched for the materials of the pyramid as well as the mechanism of how it works (Pepper’s Ghost with ray optics). Using my research for the previous week, we agreed on the OV7670 as our camera and integrating this to our design. Breyden and I looked into the specs and the pins connections to flush out our solution to this problem better. Notably, the camera gives us a constraint/ requirement of GPIOs pins that we must have for our FPGA board, which is around 18pins * 4. The camera also provides us with the FOV, 25˚, which is needed in our computation for the size of our studio. I looked into the measurement of the monitor also and we decide that a 55” TV would work after calculating the dimension required for the 4-5x enlargement of our object. Using this information, I created some of the slides for the presentation.

My work on the presentation mainly includes the creating System Overview diagrams, diagrams of the construction of the pyramid and the setup for the live studio. I helped with some of the calculations for the physical designs (the live studio dimensions, the dimensions of each picture frame, the dimensions of each pyramid panel), as well as talk with Breyden about the System Overview design. I also helped with deciding on the FPGA board that we will be using through weighing our trade-offs between available/required logic elements and GPIO pins.

Also, I spend some time rehearsing the presentation. I also met with the professor to go over our slides for this presentation.

Breyden Wood’s Status Report for 2-27-21

This week, I worked on several things. The first thing I did was help Grace prepare the group presentation given on Monday (2/22). We worked on and finalized the PowerPoint slides together as a group, and then individually reviewed a sample practice video that Grace put together of her running through the presentation. After the presentation was done, I focused my work back on researching specifics of our design, specifically the FPGA and how we were going to get video output working. Our current plan is to use the VGA output of an FPGA to display to a TV screen with an output resolution of 720p to maintain a sharp image at all times. However, we have two problems we have identified we may run into with this. The first is to do with the pixel clock. A 1280×720 image running at 60Hz requires a pixel clock of approximately 75MHz, which is high enough of a clock that we would likely need to implement PLLs to generate a faster clock to drive the display. Additionally, a single frame at 720p requires a 3MB frame buffer, which may exceed the amount of onboard memory available to us on the FPGA.

To resolve this, I have been researching fallback plans to increase visual quality in the event we have to settle for a lower resolution such as 480p. Our first plan to mitigate this is to have a robust sharpening filter, which is part of our current plan for one of the image filters inside the FPGA’s ISP. However, even with a strong sharpening filter, the outputted image would still be 480p. One way to get around this is to use a VGA to HDMI adapter, and then run the image through an HDMI upscaler. This has the benefit of shortening our VGA cable length (long cables can introduce noise into the analog signal), while dramatically boosting visual quality with the upscaler as many of these use advanced post-processing to increase quality along with resolution (versus simply stretching a 480p image to 720p or higher). One such product I have found is the Marseille mClassic (originally designed for boosting visual quality from game consoles), which reviews claim can take a 480p source to near-1080p quality (see photo below from the mClassic website).

Into next week, I plan to continue researching how we can get a high-resolution feed out of an FPGA and the potential of these fallback solutions, as well as selecting a specific board that we can begin development on. This mirrors our current work plan for this upcoming week (research), and will keep us on track for success in our project.

Jullia Tran’s Status Report for 2-27-21

At the beginning of this week, I worked on reviewing Grace’s presentation to prepare for the Proposal Presentation on Monday. We discussed pacing, and the points we should mention for each of the slides. Later this week, I have researched some of the materials for constructing the pyramid. There seems to be many ways to build this pyramid such as glass. However, after considering the size that we want this to be constructed as and how it would be very difficult to construct and cut into. It seems that the other way, which might be what we will be pursuing for our design would be to use plexiglass panels since these are cheap, light and comes in big sizes. As shown in this link here, this material maintains the holographic effects, while has the potential to be scaled bigger.

I also looked into some cameras options for our design. At first, we have the options of choosing between USB, NTSC or VGA protocol to interact with our board. We think that between the three of these protocols, VGA might be easier to implement. I looked up some of the potentials options for VGA cameras and the OV7670, OV7725, or OV5642 all seem to be very viable options to meet our budget, as well as the specs for our requirements. The OV5642 seems to be much more expensive than the other two, however with 60fps support in the specs sheet. The OV7670 only has 30fps as mentioned in the specs sheet, however, it seems that we should be able to clock it up to 60fps. This camera model also seems to be more common amongst people who use FPGA with this camera as comparing to the other two camera. The OV7725 also seems to support 60fps but not a lot of people seems to have used them. Because of this, the OV7670 seems to be a viable solution for our requirements due to its cost, and widely used by people with FPGA interactions.

Grace An’s Status Report for 2-27-21

At the beginning of the week, I prepared for the Design Review Presentation. I added animations to the slideshow and rearranged some of the information in the slide deck to allow for a clearer presentation. With the help of my teammates, we also revised our Gantt schedule to have finer grained tasks than our original schedule. Following these steps, I practiced presenting a few times, with my teammates offering very useful feedback. Then I presented on Monday, and our team subsequently responded to and discussed the provided feedback. (This is further discussed in this week’s Team Status Report here.)

Aside from the presentation, I also spent this week starting my next scheduled task: researching image filters. I have found information regarding filters used for image sharpening, where different kernels can provide different effects. Experimentation seems useful to determine which one would “best” provide local contrast, although an unsharp mask would work be reasonable as a starting point. As for chroma-keying, the best choice of backdrop color depends on what objects our live studio will be optimized for. While green is particularly good for digital cameras (to avoid visual noise), a blue backdrop would work better to avoid color spill on subjects and improve detail retention.

My progress is on schedule, as this week was assigned to working on the Design Review Presentation. Over the next week, I will finish researching these given topics, producing a summary for all the researched information that will aid in the design and implementation of the pyramid and image filters.

Team Status Report for 2-27-21

This week, we prepared and then presented our design review presentation at the beginning of the week. Following the presentation, we received helpful feedback regarding the design of the live studio used to capture video feeds of an object. A key concern is how make it possible for an object could be rotated and shown easily, without obstructing the view of the object (as moving it by hand would do). Our possible solutions include the following: a rotating disk at the bottom of the live studio, and green tweezers that would be removed by chroma-keying. We are considering having different studio setups based on what object should be shown, and we plan to spend the next week finalizing our studio design.

Another issue we have come to realize is that our current schedule is not designed to have our MVP by the midpoint demo, but rather to have the full design only done by the end. We plan to rearrange our schedule in order to better meet the midpoint demo, which likely requires building the holographic pyramid and live studio earlier than currently planned, and building the image filters later.

Thus, going forward, our largest challenges are both finalizing the design of our studio and rearranging our schedule to better fit our needs. As our next week is currently accorded to “research”, we have the time to make these changes without jeopardizing our current or revised schedules.

Grace An’s Status Report for 2-20-21

This week, I worked out the proposal presentation and details about our project idea. To do so, I contacted our assigned professor and TA in order to meet outside the regular one, in order to properly understand the expectations of our proposal presentation and our capstone project. Jullia and I attended these meetings, and we discovered issues (that we later fixed) in our project presentation, e.g. our requirements being too design-focused and too technical (and not understandable to a layperson). During these meetings (including the regular team meeting), I took notes so that the entire team could reference these notes after the fact, as we were working out our project. With my team members, I also reworded the words on our presentation in order to make the bullet points concise and easily presentable and helped rearrange information to make it organized and easy to understand. Furthermore, Jullia and I wrote down a task list and transferred it over into our current Gantt chart. I also helped with setting up our website, and I will soon post the Introduction and Project Summary after this status report. Our progress is currently on schedule.

Over the next couple days, I will be working on practicing the proposal presentation with my teammates, as I will be presenting over the next week. I may also work on slide transitions and further reorganization of information to maximize the quality of our slides. Following these presentation details, nearer to the end of the week, I may start researching image filters (chroma-keying, brightness, and contrast) as well as the pyramid design.

Jullia Tran’s Status Report for 2-20-21

Over the course of this week, I helped Grace work on the proposal presentation, specifically the solution approach and the system overview where I also discuss with Breyden to come up with some of the details for the design. After finishing the presentation on Thursday night as a team, Grace and I met with Professor Kim for feedbacks before our presentation on Monday. One of the biggest feedback was on the technical challenges slide, where we didn’t focus only on the biggest hurdles. We adjusted this slide accordingly, and also the division of work slide. We also finalized our Gantt chart. This up coming week, I hope to help research more about the different cameras available that have an interface with the FPGA so we can start ordering the parts. I also hope to look in to the use of PLLs with Breyden so we can think more deeply about our implementation design.

Breyden Wood’s Status Report for 2-20-21

This week, I put most of my effort into working on the presentation, specifically most of the image quality requirements and defining the metrics for latency and performance of the system. I have a strong background in digital photography, and I was able to provide that knowledge to the team by defining concrete metrics for how we can test and measure image quality and sharpness as we apply filters and set up the cameras. I also know a fair bit about displays and framerates, so I was able to apply my knowledge to defining requirements for frame stability and pacing. My progress is on schedule, as my group and I are nearly finished with our proposal and are preparing to present this upcoming week. This next week, I hope to help Grace give the presentation and begin ordering the materials so we can start construction of the HoloPyramid soon after.

Team Status Report for 2-20-21

This week, our group worked on creating our proposal presentation for our project. We’ve met several times both inside and outside of our mandatory lab time to discuss the content of the proposal, work on it as a group, and get feedback from the professors. Through this, we were able to come up with a concrete schedule and plan out our task list as seen below. Some of the biggest challenges we’ve had to face this week have been fleshing out some of the smaller details of our project, including the choice of hardware (FPGA vs Pi), the size of the projection, and how to quantitatively measure visual quality and sharpness. Through researching answers to these, we have been able to better understand both our project idea and our proposal itself. Going forward, we expect our next biggest challenge to be choosing the specific hardware we want to use to build our pyramid.