This week I focused on getting the keyboard overlay to closely match a physical Mac keyboard in perspective and on encoding that geometry into better user defaults. Using the phone stand, I set up a perspective test where I put a Mac keyboard in frame, mounted my phone with our app open at roughly a 60-degree angle, and rested my hands on the physical keyboard to run F/J calibration. I then tuned the Height and Top/Bottom sliders until the rendered overlay lined up with the real keyboard, set those tuned values as the new defaults, and adjusted the slider ranges so that the defaults sit roughly at the midpoint. While doing this, I realized my previous model only squeezed the keyboard horizontally, so I added a per-row vertical scale derived from the same top-bottom ratio: with 5 rows there are 4 steps from bottom to top, so I define a factor such that a^4=topShrink, so a=topShrink^(1/4), and the row heights become [1, a, a^2, a^3, a^4] for the number, QWERTY, ASDF, ZXCV, and modifier rows. This gives us a consistent perspective in both width and height. I tested this repeatedly with the physical keyboard and also refined the offsets between the letter key rows now that I have a physical reference. Below is a picture showing the overlay on the physical keyboard after the adjustments, you can see the overlay (specifically the number and letter keys) very closely matching the physical reference.
I am mostly on schedule for my part of the project in terms of overlay geometry and tap decision, since the perspective test and the new per-row vertical scaling are now working.
Next week, I want to hook my key-under-fingertip mapping up to Joyce’s tap detector so that instead of just labeling fingers, my module will take her tap events and return a concrete key label, which will then be fed into Hanning’s system so their component receives a stream of keypresses from my overlay. I am also considering adding a third slider that lets users adjust the rotation of the keyboard overlay (default 0 degree, horizontal), even though calibration already encodes tilt via the index fingers. In practice, it is easy for the two fingertips to be at slightly different depths, which leaves the keyboard a bit tilted, and right now the only fix is to recalibrate and carefully match fingertip depth. A small rotation slider would let users straighten the keyboard without redoing calibration once they are already happy with the size and position.