Author: gean

Over the past week, I worked on final logistics. As planned, I helped finish up the final presentation at the beginning of the week, fixing up slides so that they had less text and were more readable as well as more visually appealing with pictures of the pyramid. I also checked through the final presentation slides to affirm the content and helped Breyden shoot some footage for the final video and perform testing of our system. A link to the presentation slides can also now be found on our website here.

At the end of the week, I worked with Breyden on an outline for our final video, determining how long we will speak throughout different sections. I also put together some slides and recorded audio for the sections of the video discussing the background removal algorithm, display, and pyramid. Over the next few days, I plan to record more footage of our project with Breyden’s help, in order to have B-rolls so that Jullia can edit together our final video. Our goal is to get good footage of stationary and moving objects inside the studio and also demonstrate the user experience of using our HoloPyramid project. Once this is done, we will upload the final video, make a poster presentation, and finish up our final report. My progress is on schedule, and I am on track to finish our project over the next week. As this is the last status report, there are exist no further deliverables.

Over the past week, I have worked largely on the project presentation, transferring information from the previous slides to the final project presentation and adding new information. While our final presentation has a significant amount of information that was covered in our proposal presentation (the application area and solution approach) and the design review (hardware and solution description), our final presentation has the results of our preliminary testing and significant discussion of design trade studies. For example, LED strips on the side of our live studio give significantly more even lighting than point lighting.

This week, I also assisted with integration and assembly of our project. I cut out different felt squares and strips for the backdrop of our live studio and assisted with assembly. We tried out black, bright green, dark green, and dark blue felt backdrops. The different colored backdrops had different results on the background removal algorithm due to how uniformly lit they could be, and the brightness of the backdrops affected the cameras’ autoexposure settings, affecting how the object was shown. Unfortunately, brighter backgrounds were less uniformly lit, but darker backgrounds badly influenced the cameras’ autoexposure settings, causing the object to be whited out with very few details shown. We settled on dark green felt to serve as a backdrop, and we also blocked the bottom LED strip lights to better improve the even lighting of our studio. I also assisted Breyden with the adjustment process of background removal and visual quality of our product.

I am track with our schedule, as I have both worked on the final presentation and helped out with integration and preliminary testing. Tomorrow, I will listen to Breyden’s practice run of the final presentation to give feedback, and I will also assist with acquiring footage of our project to be used for the final presentation. Over the next week, depending on time constraints, I will also have the additional task of assisting with finishing tests, adjusting our project, and making the final video.

Over the past two weeks, I have finished the desired image filters (chroma-key, contrast, and brightness). I wrote a chroma-key implementation in SystemVerilog that takes in two 16-bit pixel values and a threshold determiner and outputs either the original 16-bit pixel value or zero depending on whether the provided pixel value is sufficiently close to the provided “background” pixel value. This implementation also uses Quartus’ synthesized nine-bit multiplier modules on each “color” in the 565 pixel values. The specified background color and threshold values also allow the chroma-keying module to work with the FPGA hardware switches such that the removed color and the sensitivity of the chroma-key filter can be set dynamically after hardware configuration. I have also finished contrast and brightness modules, although that would be tested at a later date (when the entire studio is integrated with all four cameras and the FPGA). It is likely that the contrast module would require using decimal multipliers on the FPGA.

As Breyden has the FPGA and cameras in his possession, he very kindly tested and debugged my code (which had an LSB and MSB mix-up error) and improved the threshold handling to be more sensitive across the desired background ranges. The effect of our image filtering modules can be seen in his status report this week.

I am on track with the schedule as all image filters are more or less finished. Over the next week, I will work on the final presentation and the final report as we finish up our project. I may also help Breyden with the integration process and/or the testing process depending on logistical details.

Over the past week, I have continued to purchase items needed for the live studio, including additional colors of construction paper (in order to experiment with chroma-keying with different background colors), a VGA-to-HDMI converter (to connect the FPGA output to the TV), and additional OV7670 cameras (in case any get damaged during the integration process).

Throughout the week, I continued to research de-noising filters that could be implemented on an FPGA, such as a median or low-pass filter. Unfortunately, both of these filters require operating at multiple pixels at a time, which is extremely difficult alongside the memory constraint. Fortunately, as mentioned in the Team Status Report this week here, we do not need a de-noising filter because the live studio’s lighting is sufficiently bright as to render chromatic noise largely nonexistent.

In addition to finishing software implementations of chroma-keying and brightness and contrast filters, I have also written hardware descriptions (in SystemVerilog) of modules that support chroma-keying and brightness and contrast filters. Given Breyden’s suggestion about the FPGA’s hardware switches, I have designed these modules to flexibly take in different values of input.

My progress is on schedule, thanks to Jullia kindly taking over the task of constructing the live studio. Over the next week, I will fully test, debug, and synthesize implementations of chroma-key and brightness and contrast filters. I may also assist with creating a cardboard top over the pyramid that is more professional and appealing than the existing one. Aside from that, I will assist with the final testing and integration that represents our last weeks of capstone.

This week, I worked on constructing the final full-size model of the holographic pyramid, which is currently at the level of detail needed for the MVP and interim demo, although we will make changes to make it clearer and more visibly professional for the final presentation.

One unexpected issue was that unlike the smaller prototypes, the full-size model’s walls sagged slightly at the center of the edges, which distorts images reflected off of the sides. Thanks to Jullia’s excellent suggestion, I cut out and taped a cardboard square above the pyramid, which adequately straightens the sides of the pyramid. This cardboard top also has the benefit of dimming the light immediately under the pyramid, which will improve the visibility of reflected images. For our final model, we will improve our pyramid by painting the cardboard black.

Figure 1: The holographic pyramid adjacent to its 1:2-size model. Despite the 1:2 size model’s perfect proportions, the full-size model’s walls sag slightly, especially in the center.
Figure 2: The holographic pyramid with a cardboard top. By virtue of being taped to the cardboard square, the pyramid’s sides sag significantly less.

Also during this week, I also implemented our desired image filters (chroma-keying, brightness, and sharpness) in Python. This enables customization of constants (such as the aggressiveness of the background removal, increase in brightness, etc.) and testing out the filters on the camera images before implementation in hardware. An example of the filters (not using output from the OV7670 cameras) is shown below:

Figure 1: The original test image
Figure 2: The test image through chroma-keying, brightness, and sharpening filters. The chroma-keying software implementation was hard-coded to remove the specific background color in the image.

I am on track with the updated schedule, in which software implementations of the image filters was moved to this past week, and the live studio construction was moved to next week. Over the next week, I will construct the live studio, possibly with Jullia’s assistance (depending on how busy she is with further hardware implementations). Additionally, I will also finish up the software implementations of the image filters by running them on images captured from our OV7670 cameras. I will also research and implement simple algorithms for noise removal in software to work with the OV7670’s chromatic noise, such as by using a low pass or median filter. It is likely that our sharpening filter may be replaced with a noise removal filter instead, although our final design may include both.

Two weeks ago, I spent most of my time on the Design Review Report, specifically writing up sections such as the Arduino and FPGA sections of the architecture overview, design trade studies comparing alternate FPGAs and analog cameras for our project, and the budget and work distribution. I also helped revise the final paper, improving conciseness.

Throughout the past week, I have found that nearby stores (Home Depot, Hobby Lobby, Lowe’s, etc.) do not have the size nor optical qualities of plexiglass that we desire, so I ordered acrylic sheets online. Because online shopping precludes the ability to verify the quality in-person, I ordered a variety of plexiglass sheets to see which provides the best properties. Due to shipping times, not all these materials have arrived, and I only received three 24″x36″x0.03″ PET sheets very late this week. Using these sheets, scissors, and masking tape, I have been able to make the 1:2 model shown below:

Figure 1: Over a pre-baked video, the 1:2 prototype displays a satisfyingly convincing three dimensional illusion.

From this 1:2 model, I have found that these PET sheets are finally reflective enough to make a holographic illusion. The 0.03″ thickness is also thin enough to be cut with scissors, which is much easier than with a plastic cutter (although not more precise) and is also sturdy enough the 1:2 model holds together well. There is a very slight (but still noticeable) doubled reflection that presents a concern, and we may want to consider a better attachment method (e.g. optical tape or acrylic glue) to make the final product more professional.

I am running very slightly behind schedule in that I have not completed the entire pyramid yet, since these materials arrived only today. However, I am certain to complete this task early next week, since the 1:2 model was surprisingly easy to construct. I will also investigate methods to improve the reflective quality of the pyramid, such as by using optical tape, suppressing doubled reflections, and/or polishing the plexiglass.

A greater concern is whether the live studio is able to be constructed over the next week, as this requires both materials and verification of the cameras’ field of vision and color capability, and there have been persistent issues with shipping times and the OV7670 cameras. I will definitely order the necessary materials (studio lights, backdrop, etc.) over the next week, but it is possible that I may switch over to preemptively working on image processing, in which case that will be the additional deliverable I will work on over the next week.

This week, I spoke with our TA to approve purchases and subsequently submitted the order for several necessary purchases: a daughter card for our FPGA board and jumper wires to connect the Arduino and cameras with the FPGA board.

Furthermore, I checked the material needed by our eventual pyramid by visiting the physical location of Home Depot to check the acrylic sheets they had in stock, in order to verify sheet thinness and stiffness. The material needs to be thin, to avoid doubled reflections, and stiff, to avoid distortion. I ordered 11”x10”x0.50” clear, non-glare acrylic sheets and a plastic cutter to prototype the pyramid. After receiving the materials, I built a 1:5.85 scale pyramid prototype with the acrylic sheets and masking tape:

Figure 1 (on left): Acrylic trapezoids cut out with a plastic cutter, mid-assembly with masking tape
Figure 2 (on right):Demo of prototype over a pre-baked video.

From this holographic pyramid prototype, several issues have become immediately apparent:

1. This is definitely not the material we will be using. It is not transparent enough and the resultant reflection is extremely blurry. This may be an issue with the non-glare nature of the acrylic, and this means that Home Depot does not stock the material that we will need, since they did not have other kinds of acrylic. We will need to find either clearer acrylic sheets or switch to glass.
2. Cutting with a $5 plastic cutter is time-consuming, even with thin acrylic sheets and small shapes. It took me “practice” (otherwise known as several misshapen trapezoids) and a few hours. My cuts were also imprecise. For our final product, we may need to either purchase pre-cut material or take advantage of the laser cutter.
3. Masking tape is also not the material to use to assemble a pyramid, as it is overly flexible and obscures parts of the holographic pyramid. Glue would work better.
4. Our requirement for “office lighting” will be difficult to meet. The blurry holographic illusion only appeared in low lighting, and this means that our eventual pyramid will likely require something on top to artificially dim the pyramid to keep the reflection visible.

My progress is on schedule in that I have built a prototype for our eventual pyramid. My progress is not on schedule in that this pyramid fails to prove that we have picked the correct material or the correct stratagem for cutting and assembling the pyramid.

As my teammates plan to work with the cameras next week, I will switch from that task to instead find the correct material and strategy to make the pyramid, so that we will be on track with assembly of the physical hardware by the midpoint demo. I will do this while working on our design review report.

This week, I largely worked on the design review presentation, and I also ordered the necessary parts for our project, speaking to our TA and professor as necessary. I transferred information between the previous proposal presentation to our new design review presentation and then updated the information as necessary. Among the various slides, I chose numbers for the validation tests that previously did not have them, included information on my researched image filters, and organized the information on FPGAs that my teammates spent a substantial amount of time analyzing and discussing (this is discussed in both of their status reports this week). I updated our Gantt Chart to better keep track of class logistics, and also ensure that our MVP would be done first, and then our other goals (such as the image filters) would be worked on at the end. Using this schedule, we will better be able to know how many logic elements our project needs, so that we know how many are available for the later image filters. This will also ensure that we plan to have a working project by the midpoint demo.

I also arranged a meeting with the professor on Thursday for the team to speak with him about our design review presentation; we received some feedback on the presentation itself, but mostly feedback on our general design and project process. As a team, we need to ensure testing the strength of our eventual holographic illusion, as part of our final validation step.

I am on track with our previous and our updated schedule, having researched the image filters by this juncture. Over the next week, I will determine how to purchase plexiglass for the pyramid materials and begin constructing pyramid prototypes (scaled down versions of our pyramid design) to check our specifications for the pyramid are accurate. I will also assist my teammates with their assigned responsibilities with the DE2-115 board and OV7670 cameras, as that is currently our most critical concern.