This week, I researched platforms for hosting our UI and looked into Tobii Eye Tracker 5 integration and setup.
We intended to use the Tobii Eye Tracker 5 Camera for eye tracking on a digital page, however I discovered some issues that may arise with integration. This eye tracker model isn’t compatible with MacOS, which is my development OS. I looked into 4 other solutions:
- Tobii Pro SDK can give us raw data and is mac compatible, but is incompatible with the Tobii eye tracker 5. All compatible devices aren’t in budget.
- The iPad true depth camera. I found an open source project that investigated eye tracking using the TrueDepth camera on an iPhone X. This will be challenging because I don’t have experience with Swift programming, and we are limited by tablets that already have a TrueDepth camera. This eye tracking may not be robust enough for our intended use, however, this solution remains accessible to musicians who can’t operate a foot pedal during performances.
- We can pivot to face tracking rather than eye tracking. Now, several open source APIs become available, and I can build out a real-time iOS app in Swift. The potential problem with this approach is that users will have to use their head to turn the page, which could result in a loss of focus. However, this solution maintains accessibility.
- The eye tracking community recommends Talon Voice’s software support for MacOS development. Here, there is good support for eye tracking, and head tracking is used to further refine miscalculations.
Regarding the UI, there are a couple of options: native iPad apps, or building and deploying a web application on a tablet. Mobile/native apps can provide benefits such as being faster and more efficient, which could have a huge impact on our real time system. Ultimately, this decision rests on which platform we collect eye data from.
Progress is on track as of now. For next week, I hope to finalize the tech stack and parts list I’ll be using and begin implementation of the UI.