This week my focus was mainly directed towards understanding the 3 texturing approaches I outlined in the previous status report and experimenting with AR plane features within our project. Since shader programming with metal is a rather herculean task, I mainly honed in on scenekit’s material api’s. I worked on updating the initial object placement app to learn how to extract a plane from the world instead of an anchor point by using the type ARPlaneAnchor instead of VirtualObjectAnchor. Using scenekit’s plane I created a plane and by read up on SCNMaterial to understand how to instantiate a texture and what properties are available to us. I went with a minimal example to create a plane just in the world and then added properties to make it translucent and attempt to map it to the worldplane. The first example is not fit well and was opaque. I managed to improve the anchoring of the plane to get it coplanar with the floor but it’s not staying within the boundaries which is something I’m meeting Nathalie tomorrow to sync on our AR progress and brainstorm approaches to anchor the plane in the most accurate way that is concordant with the tests we outlined in our design report for plane fitting. Right now I’m using a hit-test which is returning a transform that captures position and orientation to the tapped point and I think this is overfitting the plane to the tapped point rather than recovering information about the plane the transform is encoding. One idea I plan on exploring on Sunday is having the user tap multiple times to allow a plane to be extracted from the set of world transforms.
In terms of timeline I think we’re generally on track, the plane detection was earlier in our schedule but the progress we’ve made in terms of relocalisation has contributed to the camera angle memory task.
Next steps entail trying to get the planes to fit exactly and working on erasure and then building the app’s features out for human use. Professor Kim wanted to see an example of our AR apps in action so we will also be ready to show the features that we have so far.