As an initiative to make museums and art exhibits more interactive, we propose to build a 3D reconstruction system through a pin art board, called LivePin. This device will allow a user to place any 3D object in front of a camera in order to reproduce the image as a 3D impression on a pin art board. Before you would have to physically press onto a pin art board to make it move and create an accurate impression, but now with our system, all you need to do is hold something in front of a camera and the pins will move automatically.
Over the semester, we implemented the end to end system from the depth sensing pipeline to custom microcontroller based actuation control and a motorized pin display powered by servo motor and rack and pinion mechanisms. Through iterative prototyping, we explored and validated several mechanical and electronic architectures. Such exploration culminated into a CV based, motorized pin actuation display. A depth camera is used to collect depth data on an object. A computer vision algorithm converts this depth data into a depth map to dictate the heights that the pins actuate to. Pins were actuated row by row with a carriage containing 32 servos each powering a rack and pinion mechanism. The gantry moving and holding the carriage was modeled after 3D printers.
Existing display boards utilize a similar concepts (like flipdot displays), but there is no known device that moves pins automatically to create three dimensional topological designs. LivePin demonstrates the feasibility of such device while under resource and cost constrains. Our implementation shows that even a nostalgic toy can be transformed, using computer vision and embedded systems, into an engaging and interactive display.