Author: tcockrel

This past week has been spent working on optimizing the height map to 3D model algorithm and finishing upgrades to the manipulator. All that’s left to do is characterize the limits of our system and gather data that compares our scans to a traditional 3D scanner’s.

Theo finished the linear bearings upgrade to the manipulator and has made a new design for the electronics housing.

Yon worked on printing some parts including the test object, and writing the object comparison algorithm.

Sophia started testing and bug-fixing code for the automatic import of the height map into an object in Blender. So far it creates the plane, but it’s having trouble to find the correct file path for the height map. It should be able to be fixed before the final demo, but worst case scenario it takes maybe a minute to manually create the plane and add the height map to create the object in Blender.

Unit tests for the manipulator included measuring out precise rotations and making sure the manipulator could be placed on and rotate objects of different heights and sizes. We’ll continue to test this through our system test.

There are two main areas of the image processing system that need to be tested: image alignment and model creation. Image alignment can be easily tested by running several different shaped objects through the system and ensuring the resulting bounding box represents aligned images. The model creation part is a little harder to test, and is the reason we Yon has been working on the test object and model similarity score algorithm, which will general a similarity score between our scan and the ground truth, along with a quality score comparing our scan with the benchmark scan of the same object. We can only really test this on an object for which a ground truth exists, so this won’t exactly be a ‘unit test’. We will also run scans on a variety of different objects and perform a visual inspections on them.

Unit tests for the software include running the files individually to make sure they completely worked before adding another file to the system pipeline. Ex. we ran the dotNet C file (which runs the scanner) first many times, then ran the Main.py file which would include running the dotNet file, then running the align file repeatedly to  make sure it works, then added in the align file to Main.py and ran it many times to make sure it works, etc. So it was an iterative process of running individual files, then adding them to the system after they’ve shown they worked individually, then running the system to iron out integration.

Our overall system test consists of:

1. Place object on scanner
2. Place manipulator on top of object
3. Plug in manipulator and scanner to user’s computer (both USB)
4. Run project script. The system alternates between scans and rotations until the object has been rotated around to its starting position.
5. The image processing automatically derives the normal map of the object and then generates a height map.
6. The height map is imported to Blender for the user to inspect and interact with.

The only work from the user is during the setup. After starting the script, it only takes a minute or two (depending on the DPI of the scan, a modifiable parameter) for the object to be viewable in Blender.

This past week, the most significant problems we’ve encountered have been with the quality of the scans. As mentioned in Theo’s status report for this week, the stepper-suction adapter casts a black shadow behind the object that shows up in the normal map, and the rotation is still misaligned. The new mount has tighter tolerances that prevent it from sliding around, but it is still extremely difficult to place the suction cup directly in the center of the object (in order for each vector in the normal map to align). We’ve 3D printed a white stepper-suction adapter to see if that will help (it ought to blend in with the white acrylic cover sheet), and we’ve started looking into a software-level solution to the scan alignment issue. As detailed in Yon’s status report, we have two approaches to solve the alignment issue which we will test this coming week. These create very minor changes in our system, by introducing a small amount of complexity in the ‘image alignment’ step. If the image alignment goes well and creates better normal maps, we will then be able to move on to ensuring the height map turns out well, since the results are currently skewed from the misaligned normal map. There is a normal-to-height-map module currently as detailed in Sophia’s status report, but we may need to look for alternatives or edit it for our use case of small objects, since this module was normally used on larger cases.

As we continue to improve the manipulator, we’re also looking to begin testing with objects other than a quarter. Yon has designed a test object that we can start using when we’re ready, and the manipulator ought to be able to work with any object that fits in its 6inx6inx1in space. This also would be the validation of our system as a whole. When we compare the scan we take from the model, the original model we made, and the scans that commercial 3D scanners take, we will be able to calculate how close our scans are versus commercial scans using Hausdorff distance.

A scan from our current system and a normal map generated from it and its rotated counterparts. 

This past week was focused on testing the integration of our subsystems. We continued working on the Mac incompatibility issue but haven’t made progress yet. We’ve tentatively pushed it to the back of our task list, since it’s the least essential thing we’ve yet to do.

While Theo completed the manipulator and started testing for any issues, Yon and Sophia implemented the mapping code with the current control code. We were able to run a test where we ran an entire “scan,” with one of us rotating a coin on the scanner between each scan. Even with imprecise rotations and a scanner DPI of 100, we still observed decent a output map. Since the manipulator is completely working (besides some upgrades and adjustments mentioned in Theo’s status report), we will be ready to integrate everything we have so far on Monday.

We are still on track, with the next steps being to characterize and improve the manipulator while implementing our code in a blender plugin.

Since we’ve all been on schedule so far, we took Spring Break off. During the last week of February, we made progress on our prototype and have moved closer to system integration/testing.

Theo built the prototype’s structure, and we’re now just waiting on the suction cup and its mounting components to finish it. See his section for specifics and a photo of the prototype on our scanner. He also made some basic serial code for sending rotation and pumping commands, as well as python code for interfacing over serial. He’ll be working with Sophia to integrate a better version of this in the final control software.

Besides the serial code, Sophia did trial tests of the flatbed scanner with the controller software and found that it had issues with no clear solution. After many attempts and guesses at what was going wrong with the scanner to computer communication, and Theo after attempting to setup the project but finding it incompatible with newer versions of Linux, she’ll be pivoting to a more modular approach that will use unique and more compatible libraries for each OS, keeping the individual OS scanning processes in separate files. This will mean a more incremental approach to ensuring each OS works well without jeopardizing the states of the other OS’s and it won’t rely on an outdated dotnet framework (NAPS2 needed version 4.8, was only supported at all to version 8, current version is version 9).

Yon finished working out the math for normal map construction and detailed his findings in the design report. He also identified 3D scanners we can use for qualification, which gives both Yon and Theo some work next week in designing and manufacturing a test object. Now that a basic python version of the normal map code is implemented and can be used for testing Theo and Sophia’s subsystems, Yon will turn to implementing the normal map math in the design report. He also still has to identify a normal map to depth map pipeline, which can be achieved either in a custom implementation, a C library, or externally through another blender plugin / tool.

A was written by Theo, B was written by Yon, and C was written by Sophia.

Part A: On the global scale, our project has possible uses in the fields of archeology, history, and design. There are no limiting factors in who across the world could access/utilize flatbed 3D scanning besides a possible language barrier in the software (which is easily translatable).

People not in the academic environment would be more likely to use this as hobbyists who want to get detailed scans of their art, sculptures, or other detailed objects. There is a small subculture of photographers who use scanners for their high DPI, and such a group would likely be eager to map their hobby into a third dimension that they can interact with via Blender or other .obj-friendly software.

There is an emphasis throughout our project on making the scanning process user-friendly and hands-off. While this is mainly meant to accommodate repetitive data acquisition, less tech-savvy people would only have to deal with a few more steps than when using a flatbed scanner normally (place manipulator, plug in cables, run software).

Part B: Our project implements a cheap and accessible way to 3D scan small objects. One significant area of application for this technology is in archeology and preservation where cheap, quick, and onsite digitization of cultural artifacts can help preserve clotures and assist dialog and discourse around them.

That said, all technology is a double edged sword. The ability to create quick replicas of historical artifacts makes them vulnerable to pop cloture-ification which could manifest in cultural appropriation.

Part C: Our project uses a low power draw, which is an environmental boon. Especially considering its competitors are larger, complicated machines that would use more energy. Our project also leverages an existing technology, therefore reusing devices and not requiring buying a larger version that uses much more material and energy in manufacturing and usage.

The simplicity of our project also lends itself to environmentalism, since we don’t use any cloud storage, AI features, or other massive energy consumption processes. We don’t even use a separate battery, drawing power from the computer through USB. Open source projects like ours are also generally more sustainable than completely commercialized sources.

Environmental organisms and discoveries can even be captured for future research and knowledge using our project. Since archaeology is a key audience, it’s not a stretch to extend that into general biology. Scanning small bones, skeletons, feathers, footprints, or other small biological features would be possible as long as they aren’t particularly frail. This contributes to the knowledge bank of biological organisms, furthering science’s understanding. The Smithsonian for example has a public access bank of 3D scans, so our project would be perfectly suited to its use for small, detailed objects.