Harshul’s Status Report
This week we worked on the proposal presentation to flesh out our idea, define our scope, assess feasibility, and identify technical challenges. I worked on the Technical Challenges and Solution Approach sections for our presentation. This entailed translating our use case requirements and featureset into the technologies and tools that we would be using as well as coming up with a high level architecture of how subsystems of our device would fit together.
The key challenges identified were:
- Software Based Challenges
- Object detection to create an accurate map of the floor ‘plane’
- Erasing the map as we traverse the cleaning area
- computer vision component that would detect particulates
- Hardware based challenges
- Combining AR data collected by the phone with the information from our cleanliness computer vision module
- Identifying how our hardware would communicate with the iPhone mounted on the vacuum
With these challenges in mind I researched into the features and capabilities of ARKit, RealityKit and CoreBluetooth to understand and identify how these technologies will solve these challenges and assist us in building our application and represent how they would fit together in the diagram below. We also spent time as a team justifying why leveraging Apples Existing AR toolchains rather than rolling our own implementation with ROS and Unity would be more aligned with the non functional requirements of form factor to a product that is easy to use coupled with the vertical integration of these tools in comparison to ROS, Unity etc having interoperable but completely separate libraries for each of these purposes along with keeping costs down purchasing LIDAR sensors and using the embedded lidar within our phones.
This progress is on track with the Gantt chart and next steps involve meeting as a team to order materials and working to spin up an Xcode environment to start iterating and experimenting with ARKit. This experimentation will familiarize ourselves with our technology stack and help us create a more detailed system & technical design.