Posted on

Hanning’s Status Report for October 18

This week I added a calibration instructor and a small finite-state machine (FSM) to the camera webpage. The FSM explicitly manages idle → calibrating → typing: when a handsDetected hook flips true, the UI enters calibrating for 10 s (driven by performance.now() inside requestAnimationFrame) and shows a banner with a live progress bar; on timeout it transitions to typing, where we’ll lock the keyboard pose. The module exposes setHandPresence(bool) for the real detector, is resilient to brief hand-detection dropouts, and keeps preview mirroring separate from processing so saved frames aren’t flipped. I also wired lifecycle guards (visibilitychange/pagehide) so tracks stop cleanly, and left stubs to bind the final homography commit at the typing entry.

I’m on schedule. Next week, I’ll integrate this web framework with Yilei’s calibration process: replace the simulated handsDetected with the real signal, feed Yilei’s pose/plane output into the FSM’s “commit” step to fix the keyboard layout, and run end-to-end tests on mobile over HTTPS (ngrok/Cloudflare Tunnel) to verify the calibration→typing flow works in the field.

current webpage view:

Posted on

Team’s Status Report for October 4

We made a design adjustment: The camera needs to sit at a different angle than originally planned. To find a suitable angle for our current gesture recognition model, we’ll first use an adjustable, angle-changeable device holder to test several angles, then commit by purchasing or fabricating a fixed-angle holder once we identify the ideal angle.
Current risk we found: Mobile devices couldn’t open the HTML directly (no real web origin, only Microsoft edge can open, but blocks the camera access)
Mitigation: We now host the page on a local server on the laptop and connect via the phone over the LAN (real origin/permissions work), with HTTPS or a tunnel available if needed.

Schedule updates: Junyu scheduled an additional week for fingertip recognition shifts out because the this task takes longer than expected since we want to test multiple different methods to ensure accuracy, and fingertip detection is curial to the accuracy of key detection. Hanning’s task for week3 and week4 switches to align with Yilei’s calibration process. Yilei reports no schedule change.

current schedule:

Posted on

Hanning’s Status Report for October 4

This week I found and fixed the “mobile can’t open HTML” issue by serving the page from a local host on my laptop and connecting to it from a phone on the same Wi-Fi (instead of file://). I verified that modules load correctly and that camera permission works when accessed via a real origin, documenting the steps (bind to 0.0.0.0, visit http://<LAN-IP>:<port> or use HTTPS/tunnel when needed). I completed a basic text editor under the camera preview: independent <textarea> wired with helper APIs (insertText, pressKey, focusEditor) . I also began research on autocorrection methods, found lightweight approaches (rule-based edits, edit-distance/keyboard adjacency, and small n-gram/LM strategies) and noting how we can plug them into the editor’s input path.
I’m on schedule. Next week, I plan to display a calibration instruction panel on the webpage and push the autocorrection prototype further. There is also a slight change of my schedule—originally calibration instructions were slated for this week and the editor for next week, but I swapped them to align with my teammates’ timelines.
Posted on

Hanning’s Status Report for September 27

This week I focused on two coding tasks and one deliverable with the team. Early in the week I restructured the camera webpage code into three modules (HTML framework + camera.js + snapshot.js) Fixed local serving issues on specific browser (safari) and makes future integrations easier. I then started implementing a built-in text editor below the video preview (textarea + helper APIs like insertText/pressKey) so that we can type something into a real target. In parallel, I worked with my teammates to complete the design presentation slides (webapp part and testing part) Next week I plan to further work on the text editor and begin basic auto-correction implementations.

Posted on

Hanning’s Status Report for September 20

This week I focused on building the camera input pipeline. Early in the week I set up a webpage that requests camera permission and streams live video using the MediaDevices API, with controls to start/stop, pick a camera (front/rear), and a frame loop that draws each frame to a hidden canvas for processing. Later in the week I added single-frame capture. I can now grab the current video frame and export it as a JPEG (via canvas, with optional ImageCapture when available). Next week I plan to write some API to wire these frames into the CV part and begin basic keystroke event prototyping.