Risks
Our challenges currently lie with integrating the subsystems that we have all been working on in parallel. From our discussions this week, we have decided on data models for the output of Erin’s dirt detection algorithms which are the inputs to Nathalie and Harshul’s AR mapping algorithms. Current risks lie in establishing Bluetooth communication between the Jetson camera and the iPhone: we have set up the connection as receiving/sending and see the available device, but Apple’s black-box security measures prevent us from currently sending files. There have been developers that were able to circumvent this in the past, and so we are actively researching what methods they used. At the same time, we are actively exploring workarounds and have contingency plans in place. Options include employing web communication via HTTP requests or utilizing file read/write operations.
Other risks include potentially slow drawing functions when tracking the camera. Right now, there seems to be a lot of latency that impacts the usability of our system, so we are researching different methods in ARKit that can be used in a faster way. To address this, Nathalie is exploring alternatives such as utilizing the SCNLine module to potentially enhance performance. Similarly, Harshul is working on creating child nodes in a plane to see which is faster. We can always use GPU/CUDA if needed for additional speed up.
In addition, we have our main software components making progress but need to focus on how to design and mount hardware. This is a challenge because none of us have extensive experience in CAD or 3D printing, and we are in the process of deciding how to mount the hardware components (Jetson, camera, active illumination) such that it fits our ideal criteria (i.e. the camera needs to be mounted at the identified height and angle). Doing so earlier (immediately after the demos) will allow us to iterate through different hardware methods and try different mounts that we design to figure out what holds the most stability while not compromising image quality.
Schedule
In the coming week, we plan to flush out a plan for how to mount our hardware on our vacuum. We have already set up the Jetson such that it will be easy to fasten to the existing system, but the camera and its positioning are more challenging to engineer mounts for. In addition, the AR iPhone application is nearing the end of its development cycle, as we are moreso working on optimizations rather than core features. We are considering options for how to mount the iPhone as well. Nathalie has been working on how to pinpoint the location of the rear camera view based on the timestamps received from the Jetson. This may still need to be tweaked after we get the Bluetooth connection to be fully functional, as this is one of the main action items we have for the coming week.