This week our group was responsible for ordering components and setting up our work environment so we could begin implementation as soon as Spring Break ends. I was responsible for identifying that we were missing a couple required items in our order form, such as the microsd card, sd card writer, and power supply for our embedded system. We also considered availability of components that do not need to be ordered, such as wires and soldering irons to connect GPIO pins with the motor driver and to hook up the motor driver to the stepper motor. We will also require power for the stepper motor, which requires a 5V variable amp supply to the driver, which we need to figure out (the Jetson also takes 5V at ~2A, but this signal is rectified from a 120V AC wall signal). The last thing we want to do is burn our driver board, since it is not cheap and we would need to wait for additional delivery time and would lose an additional chunk of our budget. Because of this, we will prototype analogue circuits on a mini-breadboard (provided from the lab) with a potentiometer to ensure we meet the voltage and current requirements of the motor driver.
I have had a very busy week before mid semester break as many courses (including the one I am TAing) are rushing to gather grades for the mid-term report. I have also departed on a 10 day vacation starting Thursday morning to Europe (Norway), and will return on Tuesday the week after Spring Break, meaning I will be missing another mandatory lab session. Luckily, we have factored in Spring Break and additional margins for slack to account for this busy time of the year, and thus we are prepared to get back on top of this project after the break.
This week I have also been considering implementing the testing routines, so that by the time our prototype software has been completed we can get a sense of its accuracy and timing efficiency. Instead of this however, we first need to determine our intermediate data formats, such as what the partial point clouds look like before merging multiple scans or during a single scan. To provide unit testing on each software component in the pipeline, these formats must be determined so that our testing and benchmarking software understands how to interpret our results. Because of this, instead of writing the testing code first, I began planning the prototype implementation software first, to allow also Jeremy enough time to find ground-truth models for accuracy analysis.
When writing test prototype code, the only parts that matter to us are:
- Accuracy meeting our project requirements
- Ease of coding and simplicity
The second requirement of the prototype code is not important to the final implementation of our project, but it is important to the prototype development. This is because we do not want to waste all of our remaining resources writing code that will not be used in the final project.
To meet the accuracy goal, we will test our prototypes extensively. These same tests can be reused for our final GPGPU implementation, so we do not need to re-implement testing infrastructure. This also makes sense since the prototype implementation should be a “complete” implementation that does not fully meet our timing requirement. If we have trouble implementing the accuracy goal for a stage of the prototype, we may choose to implement the section directly as our final GPGPU optimized software.
For ease of coding, Python is unparalleled in both our team’s knowledge of it, and the amount of computation that can be expressed in small amounts of syntax. Because of this, we will choose to use Python libraries to implement the prototypes. We will also use popular Python libraries for the prototypes, since we do not want to get stuck on issues that cannot be resolved. The library we use must be open-source (to avoid licensing complications and fees), have good documentation (for ease of development), and have an active support community (for ease of debugging).
Because of these, and prior experience, we will be using the following Python libraries to implement our prototype pipeline:
- Numpy for representation and transformation of matrices
- Scipy for solving optimization problems on Numpy arrays
- OpenCV for image filters, point-in-image detection, and other calibration procedures
These three technologies cover our entire use-case. Once our prototype pipeline is completely implemented, we can use the Python library Numba to auto-generate GPGPU code from our pipeline. If this code is sufficient, we do not need to implement C++ by hand for the GPU. If sufficient optimizations cannot be found, then we will need to implement some sections by hand to meet our timing requirements.
As a summary, my next steps:
- Write a “formal” specification identifying how intermediate data structures are organized throughout each stage of the pipeline.
- Write calibration routines using Numpy and OpenCV to do intrinsic/extrinsic camera calibration using the CALTag checkerboard pattern on the rotational platform.
- Write global calibration parameter optimizationusing Scipy
- Write laser-line detection using OpenCV
- Write laser plane calibration using Numpy and OpenCV
- Write 3D scanning procedure, interfacing with the motor controller and camera in real-time. This procedure will use ray-plane intersection with Numpy. Once this is complete, we will not only have unit tests for each section, but we can do a complete integration test on a single scan (after mesh triangulation done by Jeremy)
- Write the integration testing code for accuracy. Unit testing procedures will have already been implemented during development for each of the above sections of the algorithm.