The start of my week was focused on preparing to present our design review, which happened Monday and (I think) went well! Since then I have been pretty sick, but was able to – thanks to a cocktail of basically every over-the-counter cold medicine known to man – make some progress on the image recombination algorithm.

I decided to start working with OpenCV, as it is more familiar to me and seems to be more well-documented. 

I’ve gotten some preliminary code working which is able to recombine images that I’ve broken up manually. I began with simple images, but then started testing with coral reef pictures, and was able to stitch them together successfully.

I went on to try and break images up both the vertical and horizontal directions, which failed miserably but brought up some good things for me to think about. Specifically, I realized that OpenCV only seems to support stitching together panoramic images. This doesn’t mean that we won’t be able to create a grid of n x m images, but we will have to be more deliberate about how the initial images are taken and ordered.

Additionally, when trying to stitch together 4 images, which were supposed to make up a 2×2 grid, I was given a success exit status from OpenCV, even though the resultant image was wildly incorrect. This brings to mind the need to validate resultant images more accurately than relying on exit codes.

Going into next week, I plan do the following::

• Develop and implement an algorithm that, given a folder of particularly named images, is able to recombine an n x m image.
• Implement code which more quickly can split up existing images into a folder of test images to combine, since generating tests manually is quite time consuming (I’ve developed the algorithm for this, but need to actually write the code)
• Run tests to specifically determine how much overlap feature is necessary for success
• Think about (but not implement yet) a better way to validate resultant images
  • This will be more necessary once we’re combining images taken by our robot, but for now we can just do pixel comparison to the original
• Investigate the impact of recombination (if any) to image quality
• (Likely not for next week, but another future consideration) Learn how to edit image metadata to include any necessary data/ordering information needed for recombination later

Going into next week, I am a bit ahead of schedule. This week’s work introduced a minor roadblock in that we won’t be able to automatically recombine images in 2 directions, however, because of my success with OpenCV (and not having to pivot libraries), I believe that I’ll be able to still be on schedule at the end of next week.

Below is an image of a coral reef which has been split up for recombination testing. Each green dot is a key point, which allows for overlap detection.

This week, most of my time was spent creating and preparing to present our design slidedeck. Abie and Emma have been at a swim meet since Tuesday, so I met with Prof Kim on Wednesday and together we refined some details of our design, specifically the movement of the underwater component.

For this component, I found some inspiration from Thingiverse and Printables for both stepper motor spools, and corresponding level wind systems. I will be remixing existing parts in the coming weeks to allow for the underwater component to move up and down. With the help of Prof Kim, I was also able to develop the concept for a rigid guide which guides the underwater component’s tether off of the side of the boat, allowing it to be controlled by the stepper driven spool without compromising the boat’s ability to float. Additionally, this design allows for the underwater component to be retracted to the edge of this rigid part, keeping it from dragging behind the boat during larger movements. During small and slower traversal movements, the part can safely drag behind a little bit, while the rigid part will still reduce this significantly.

Lastly, I spent time this week researching libraries (primarily Python libraries) to support the image recombination feature of our project, allowing for the device’s user to create a comprehensive map of stitched together underwater images. After researching a number of options, I’ve narrowed it down to two contenders: OpenCV and COLMAP; I’ll be writing some test code in the coming week to select the library which best fits our project’s needs.

OpenCV is most familiar to me, as well as is fast and very automated especially when there is image overlap, which we anticipate being the case. One notable weakness of this library is that it can struggle to accurately combine images which have uniform textures or objects. Without testing, we’re not sure if coral reef will be too redundant looking for OpenCV to recombine the images without significant user intervention.

I do not have any prior experience working with COLMAP, but from my research this library seems to greatly support the recombination of unordered images, as well as supports 3D reconstruction. I am interested in the 3D aspects of this library, as this is a feature we didn’t intend to add to the project, but could be a great addition if able to be done upfront and easily. The drawback of this library is that it appears quite computationally heavy, which might break some of our timing-based user requirements. Additionally, this extra computational power may not be worthwhile if our underwater image resolution isn’t good enough for 3D features to be extracted. 

Based on our gantt chart, I am perfectly on schedule and don’t know of any additional risks or hurdles that have been introduced this week!

Note: I pinky promise that I posted this on time, I just had to edit it to add the right tag and it changed the date :/