Author: tshaik

This week, I worked on integrating the homography logic into the overall Flutter UI with Caroline. This required me to make some changes to the process of calibration and the input of images. Furthermore, I added more error handling methods to make sure the program does not just crash on usage (when more than 4 corners are detected). A problem I have been working on is the light gradient in various light settings. As the light disperses with the angle of the projector, the light is considerably dimmer and faded out. To fix this, I may have to change the overall projection to be closer to the projector. Furthermore, we standardized the size of the projection space to ensure that we can work perfectly in a set space – but also remain flexible to tuning to other spaces.

This coming week, I will be spending a lot of time completing the final report and filling out the poster. Additionally, I will be adding wire management and aesthetic factors to the setup (specifically the camera mount). As needed, I will continue tuning the calibration and overall system display.

This week, I finalized a camera mount that is set to an appropriate height for our testing environment in HH 1303. Additionally, I decided to use red markers in order to guide the user in outlining the table in the case of projector overflow. Processing these sped up calibration significantly and allowed it to be a more consistent determination of the table outline. The rainbow streaking is a problem we are circumventing by adjusting the selected colors for projection and reducing the overall amount of projection. After these updates and the new camera setup, I have been working on mapping the projection appropriately onto the table so that the homography retains a standard level of brightness. This means playing with the height and angle of the projector further to find an optimal setup.

This coming week, I will finalize my process and complete the integration of calibration into the UI. This will enable us to run everything from one device and one package. In this project, I learned a lot more about calculating homographies and the limitations of a projector system. I also learned how to use Flutter, which I’ve never had to use previously. My main strategy was to rely on forums and data sheets in order to gain a full understanding of the components I was using. For Flutter, I really relied on their online documentation and example apps other people had made using this tool.

This week, I focused on putting together the entire pipeline on the actual projector and camera setup. There were a lot of lessons learned during this portion of integration because we learned we needed more stable mounting mechanism and clearer guidance toward the user on setup. Furthermore, when testing the camera with the projector, I realized that the camera saw the projection in the RGB states because of the faster fps. This will likely not be a problem however by grayscaling and thresholding. This is necessary for the auto-calibration process. The USB camera we were testing with has a fisheye effect that our actual camera does not have.  None of these factors will delay our timeline significantly, but this does mean I need to put more bandwidth on manufacturing a more secure mounting for the camera and projector.

Next week, I will manufacture the mounts for the projector and camera. I will also work on speeding up the autocalibration by running some computer vision on the images taken in.

Verification:

• Projector Calibration
  • Ensure the bleed over onto the floor is within the specified range from the requirements. If not, update table corner detection to identify table bounds.
  • Ensure that the spacing is measured to be the distances specified in the requirements. If not, change table bounds.
• Hardware
  • Ensure that the hardware mounting for the projector is secure and not susceptible to falling from minor bumps. Verified via checking camera detection of projection difference.
  • Ensure that the hardware mounting for the camera is secure and not susceptible to falling from minor bumps. Verified via checking camera detection of projection difference.
• Recipe Processing
  • Ensure that the recipe is split into appropriate steps. This will be analyzed from the user acceptance process.

This week, I worked on translating the warp homography and code to Flutter and tried it on the actual planned projector setup. Previously, I had been using a smaller projector for testing purposes. There have been several snags in the process of designing the calibration for the initial warping. Since the current method requires user input for the homography, it makes the process a little harder and less user friendly than intended. The checkerboard is meant to make this easier but is still under quality testing. Integrating with Flutter has been more difficult than anticipated given the different hierarchical structure. Futhermore, I think it may be necessary to design a lip for the new projector stand to ensure the projector is comfortably situated on the stand while optimizing the projection on the table.

I am on track this week, but plan to spend significant time on streamlining the calibration procedure.

This week, I spent some time working with the Xavier and tried to connect the camera to it so that we would be able to do some preliminary testing. However, there were a few roadblocks here that should be resolved this week as we work to onboard the Xavier. On the other hand, I revisited the projection method such that it utilizes the camera and prevents the user from having to mess with the projector setup. This would involve another step in the calibration process that would create the homography based on the existing camera and projector setup. Furthermore, I have designed the first iteration of the hardware mount that will be 3D printed this week unless an existing metal/wood solution is found. There is some concern that the filament will not hold up.

I am on track and this week I plan to work on completing the calibration procedure, mount and sorting out our central compute.

This week, I talked to Marios and acquired a much larger tripod that is perfect for mounting the projector. Since the new one is sturdier than the first, the first will be used to mount the camera + AGX and the second for the projector. This is a positive change from our previous system and will require new brainstorming on mounting designs. Additionally, I outlined the projector calibration and have begun implementing the script for it. Currently the procedure is (1) inform user calibration is taking place, (2) detect table outline through user markers, (3) verify that the projected calibration grid matches an unwarped version and either reimage in software or prompt the user to raise the projector to preset height/angle changes. Caroline and I also discussed the pros and cons of using the homography that came with the Flutter UI vs a Python script that would require a frame by frame input.

I am on track and this week I plan to schedule out large portions of time to implement a working calibration script and design a hardware mount with our new materials. Our team is on track for integration before the interim demo

This week, I placed the order for a stronger projector and received it. I had been testing with a smaller Panasonic projector, but after testing in different light modes, I found that the brightness was insufficient to meet the requirements of the project. Afterwards, I helped my team setup our coding workspace in Bitbucket so that we could track our progress by issue using Jira and regularly merge with each other to avoid conflicting code. This way, integrating everything at the end will be more seamless and we can avoid any downstream issues. There was a shuffle in task assignment so I will no longer be focusing on setting up the backend of the user site. Instead, I will be focusing more time in making sure the projector is correctly and intelligently calibrating to the workspace table. Therefore, I focused on researching potential steps for the projector calibration mode through existing calibration methods for projectors. After reviewing various resources (these 2 were particularly helpful: https://us.seenebula.com/blogs/buying-guides/projector-calibration and https://simplehomecinema.com/2022/07/06/what-you-need-to-calibrate-your-tv-or-projector/), I was able to gain more insight into the types of things I needed to ensure were calibrated for a satisfactory user experience.

I am currently on track with my tasks. Next week, I will have a barebones setup for a calibrator that I can display through the projector. Additionally, I need to think of an alternative mount for the larger projector on the tripod.