This week the team finished and submitted the design report. The biggest change from the draft was how we tell SPARK’s story. Rather than jumping straight into architecture, the report now opens by building the case for why on-device AI is a hard requirement in legal, healthcare, and compliance settings, not just a nice-to-have. That framing ended up making the use-case requirements feel a lot more motivated, since every performance target is now tied to a specific reason the product would fail without it.
On the technical side, we finalized a number of specs that were still estimates going into the week. UART baud rate is set at 921,600, which keeps bridge latency to ~13 ms per packet and takes it completely off the critical path. We also reworked the latency model to use a Time-to-First-Token constraint instead of a hard round-trip budget, which is more honest about how multi-window summarisation actually behaves, since it’s the heaviest task in the system and shouldn’t be held to the same deadline as a single autocomplete call. For the LLM, we settled on Phi-3 Mini running via llama.cpp on the Jetson AGX Xavier, with benchmark data showing 25 tokens/s output speed which satisfies our 20 tokens/s requirement.
We also added detailed diagrams for all five subsystems following feedback from the design presentation. That was probably the most concrete deliverable this week, since the architecture is now fully documented with dedicated figures rather than described in prose alone.
Next Week: The goal is first integration between the host-side Python app and the Jetson inference stack, connecting the context capture pipeline over UART, confirming SparkDB is writing snapshots correctly, and getting the full Capture-Distill-Infer cycle running end-to-end. We’re expecting to find bugs, not a working system. Formal integration testing starts the week of March 17 per the schedule.
Biggest open risk: Whether the 500 ms TTFT target holds under real workloads once the full context retrieval pipeline is active, rather than the isolated benchmark conditions we’ve been testing against. Our early integration should give us time to identify and address any gaps before the evaluation phase.
A was written by Leonard, B was written by Tatyana and C was written by Sida.
Part A: SPARK is designed to be useful far beyond the academic environment in which it was built. For a non-technical user, the barrier to AI assistance has never really been access to the technology itself, it has been the trust and setup cost that comes with it. A small business owner, a community health worker, or a local journalist does not have an IT department to configure a local LLM, and they are unlikely to read a privacy policy carefully enough to understand what a cloud tool does with their data. SPARK removes both of those barriers: it is a plug-in USB device that works without configuration, and its privacy guarantee is physical rather than contractual, the data never leaves the machine by design. This matters most for users in contexts where a data breach or regulatory violation is not an abstract risk but a concrete one, such as a clinic in a rural area handling patient records, a small law firm without enterprise-grade cloud security, or a freelance professional working with confidential client material. These users exist everywhere and share the same underlying need: powerful writing assistance that does not require them to be technically sophisticated or to place trust in a company they have never heard o
Part B: .Different cultures carry different expectations around privacy, data ownership, and trust in technology, and SPARK’s design accommodates that variation by default. In cultures where institutional trust is low or where government surveillance is a lived concern, the premise of a productivity tool that requires work to pass through a third-party server is not favorable regardless of capability. SPARK’s local-only architecture removes that barrier entirely. There is also a meaningful economic dimension: because SPARK is built on open-source models and infrastructure, it avoids the subscription costs that make cloud AI tools inaccessible in lower-income contexts. In professional cultures where confidentiality is not just a legal requirement but a deeply held ethical norm, such as attorney-client relationships or doctor-patient communication, SPARK’s design aligns with the values of the user rather than asking them to compromise those values for convenience.
Part C:The environmental footprint of AI is increasingly a point of concern, and SPARK’s architecture takes a meaningfully different position than cloud-based alternatives. Every query sent to a cloud AI provider is processed in a large data center, consuming significant energy and water for cooling at scale. SPARK performs all inference locally using small, quantized models on the Jetson AGX Xavier, which are substantially more energy-efficient than the large models running in cloud data centers. Smaller models require fewer computational resources per query, which translates directly to lower power draw and a reduced carbon footprint per interaction. The system is also designed for longevity: by running on existing hardware rather than requiring a continuously updated cloud subscription, SPARK reduces pressure to frequently upgrade devices, which in turn reduces electronic waste. The Jetson AGX Xavier was deliberately selected from existing ECE inventory, which aligns with a reuse ethos rather than driving demand for new hardware manufacturing.
