Author: sidac

• Updated the LCD UI to reflect the new button layout, ensuring the on-device display stays in sync with the revised hardware interaction model.

• Reworked the synthesis feature so that it now pulls context from relevant apps rather than only the current active app. This is a meaningful improvement to the core value of SPARK — the system can now produce more complete and useful summaries by considering the broader set of open context, not just whatever window happens to be in focus at the moment of the keypress.

• Next up: making the response drafting feature smarter, with improvements planned for next week.

• Completed final integration of the full SPARK system — all features are working end-to-end and the system is ready for testing and demo.
• Decluttered and reorganized the repository to significantly lower the barrier to getting started. This includes one-click setup scripts that automatically configure the SPARK app on a fresh Windows or macOS environment, removing manual dependency setup steps.
• Helped prepare materials and testing results for the final presentation.

What new tools or knowledge did you need to learn?

Repo Management: Clearly defining the file structure of the repo early on can significantly boost clarity, productivity, and readability. Things will get messy really quickly, especially when collaborating with others. Same with Git best practices: do commit and commit often, branch management, commit message style, etc.

Development Environment Management: same ideology. What may work on your machine may not work on your other machine, let alone your teammate’s one. This is especially true for our team: I work mostly in Windows, and Tatyana works mostly in macOS. Carefully selecting mutual toolchains and navigating the OS-specific tools is 100% needed. Sometimes, hardcoding is necessary; most of the time, it should be avoided.

Agile Development for PCB: for early prototypes(which is every capstone project – no project would REALLY pass the early prototype stage), have more flexibility. Have test points, backup vias and pads for manual soldering and rerouting, 3d modeling for spacing, mounting holes for casing. order extra components… All very necessary. When making iterations, one should try to use as much already-bought components as possible.

The learning strategy was mostly hands-on and informal: running the scripts repeatedly on clean environments to catch failures, reading platform-specific documentation and Stack Overflow threads for edge cases, and iterating quickly based on what broke. This trial-and-error approach on real machines was far more effective than any top-down reading would have been for a task this integration-heavy.

• Continued system integration work. Attempted to port the UI demo directly onto the current Pico implementation, but the integration is more complex than initially anticipated: the UI cannot be drag-and-dropped onto the existing firmware. Will continue debugging the RAW HID / UI bridge next week.
• Designed a second version of the PCB with several improvements over v1: better mounting holes for more secure physical assembly, clearer silkscreen labels for easier debugging and assembly, and modular connectors for improved adaptability.

• My work: Spent significant time debugging and testing the full end-to-end loop: Host app → Pico → Jetson → LLM server → back to Host. Solved multiple issues to when bridging the app-pico connection with the pico-jetson connection, then made the full loop work. The results need tweaking, but the core functionality is completed and ready to demo
• Major architecture refactor: Moved the pico firmware from QMK to Circuit Python. Due to the nature of the pico shifting from a keyboard to a transmission bridge between the host-side software and the LLM, the QMK firmware is capable but redundant. CircuitPython gives better development and debugging experience and is easier to work with. It still keeps the general HID functionality and can type back, acting as a keyboard if needed.
• Quality of Life:
  • Implemented live capture deduplication so the polling feed only appends text when the content actually changes, avoiding redundant entries.
  • Added robustness improvements including automatic device reconnection after unplug/replug on both the Jetson bridge and host HID client, a self-window guard so SPARK doesn’t capture its own window, a fixed-size
    scrollable output window, and a quit button.
• Pending work:
  • moving the host-side database implementation to Jetson
  • Review for the 2nd version of PCB design.

• Works with Tatyana on the communication protocol between the Pi Pico and the host-side software. Still needs refinement and further integration
• Worked on the integration between the Pi Pico and the Jetson. Successfully created a demo that sends prompts and receives responses through UART. The Jetson-side bridge program receives the prompt, makes API calls locally, and sends the response back.
• The delay was reasonable, and it simulates the feature calls from buttons.

• Created a proof of concept desktop app that’s able to communicate with the keyboard using RAW HID reports. The desktop app is able to send a large block of text or data to simulate the context we are sending. The keyboard then types back a simulated summary, which is just the first and last words of the text block. This is to simulate that the LLM inference outputs in a stream, not act like an instant paste of the entire text block.
• The next step is to integrate this communication with Tatyana’s actual desktop GUI app

• tested the newly released Qwen3.5 models, including 0.8B, 2B, 4B and 9B versions(all Q4_K_M). All tested with 8K context length(double of our design requirement. Still need to do more detailed tests on whether the models can accurately perform the tasks, but based on online reviews, they will definitely be better than the Qwen3 models.
  • 9B: read 217, write 10.84
  • 4B: read 341, write 14.88
  • 2B: read 839.2, write 30.99
  • 0.8B: read 1149, write 41.58
• Result: the Qwen 3.5 4B model is decently slower than the Qwen 3 one, which violates our design requirement. Qwen 3.5 2B, on the other hand, is faster than the Qwen3 4B.  We can set both as backups while developing further.