Posted on

Sida’s Status Report for 4/4

 

  • Continued system integration work. Attempted to port the UI demo directly onto the current Pico implementation, but the integration is more complex than initially anticipated: the UI cannot be drag-and-dropped onto the existing firmware. Will continue debugging the RAW HID / UI bridge next week.
  • Designed a second version of the PCB with several improvements over v1: better mounting holes for more secure physical assembly, clearer silkscreen labels for easier debugging and assembly, and modular connectors for improved adaptability.

 

Posted on

Tatyana report 4/4

This week my contributions spanned device-side UI development, system architecture, and the conceptual foundation of the team’s testing strategy. Most of it in preperation for interm demo.

The majority of my time was spent leading the design and development of the device-side interface for the 2.4″ LCD display. This involved planning the full layout across all screen states, prototyping each screen, and completing the button mapping for the physical controls. Working at 320×240 resolution with the constraints of the Pico required a different design approach than the host-side interface,  to keep the UI readable at that scale. To avoid blocking overall progress while the team continues debugging Jetson-to-Pico communication, I did this to make a parallel task, seprating  it from the live data pipeline until integration is further along.

On the architecture side, I drove the decision to transmit data from the host to the device in the form of SQL queries rather than raw or pre-processed output. The rationale was straightforward: offloading that structure to the host reduces unnecessary GPU load on the Jetson and takes better advantage of the processing capacity already available on the host side.

 I worked on the concept for the team’s testing plan. The core idea was to evaluate the system across two dimensions simultaneously: technical output grading against defined system criteria, and a qualitative signal from a randomized pool of users asked to grade system responses directly.

Next Week:

 I will focus on improving the web scraping layer, refining the database design, and advancing the test design. On the scraping side, the goal is to improve selector reliability and parsing robustness so the pipeline handles a wider range of real-world page structures cleanly. For the database, I plan to revisit it. And on testing, I will work to making concrete test cases and evaluation criteria ready for execution.

Posted on

Team Status Report for 4/4

This week, the team performed relatively well during the interim demo. The demonstration showed that the project is making solid progress, and the overall system was able to present its core idea effectively. The demo also helped confirm that the team is moving in the right direction while highlighting a few remaining areas that still need refinement. The communication between the Jetson and the Pico has been proven stable and full system integration is now nearly complete. Most of the major hardware and software components are working together, and the project is close to reaching a more finalized operational state. There are still a few features that are being polished or completed, but the system as a whole is much closer to the intended final design MVP.

The team’s main focus will now shift toward testing and verification. This includes developing and refining testing suites, making sure each feature can be evaluated in a structured way, and verifying that the system performs reliably under the intended use cases.

Currently, the testing plan consists of 4 grading rubrics for each of the main functions respectively and are attached below. The grading rubrics are on a 5-point scale and each feature has 6 criterions. A rough draft of user guide for the testers are also attached and will be refined later.

Next week, the team will continue finishing the remaining features while placing the main emphasis on testing and verification. The goal is to further refine the testing process and evaluate the integrated system more thoroughly.

Posted on

Leonard’s Status Report for 4/4

This week, my main focus is to develop structured testing suites for future SPARK testers. I worked on organizing the testing process so that evaluators can interact with SPARK in a more controlled and repeatable way. Instead of relying on open-ended testing, I created a set of guided testing cases tied directly to SPARK’s four main features.

A major part of this work involved defining the exact workflow for each testing case as well as estimating the timing of each step so that each case can be completed in a few minutes, making the testing process more practical. The current plan is to use two testing cases for each of the four SPARK features, giving a total of 16 combinations.

Next week, I plan to finalize the materials for each testing suite, prepare the rubric sheets for testers, and begin organizing how the testing sessions will be carried out.

Posted on

Tatyana’s Status report 3/28

This week my contributions were concentrated in two areas: device-side UI development and software architecture, with additional conceptual contributions to the team’s testing strategy.

On the UI side, I led the design and development of the device-side interface for the 2.4″ LCD display. This included planning the full interface layout, prototyping all screens, and completing button mapping for the physical controls. Given the constraints of a 320×240 display, significant consideration went into ensuring the interface remains functional and readable at scale. This work was deliberately advanced as a parallel workstream to avoid blocking progress while the team continues debugging full system integration.

On the architecture side, I drove the decision to transmit data from the host to the device in the form of SQL queries. This approach was chosen to reduce processing load on the Jetson’s GPU and better leverage the host side’s computational capacity. The decision had meaningful downstream implications for how the host and device communicate and directly shaped the current host-side packet sending and scraping interface.

Finally, I originated the concept for the team’s testing plan, which combines technical output grading with a randomized user evaluation component to assess response quality from both a system and human perspective.

Posted on

Team Status report 3/29

This week the team focused on development in preparation for our upcoming demo, with progress across both software architecture and hardware integration.

On the software side, we moved from a host-side prototyping to the a device-side execution model. The host is now responsible for packet sending and a scraping interface rather than running the full stack. A key decision this week was to send data in the form of SQL queries to the device, which reduces GPU load on the Jetson and better leverages host-side processing power. We are actively working through communication challenges between the Jetson and Pico as part of closing the full development loop.

On the hardware side, the PCB is in hand and system integration has begun. In parallel, we started work on the device-side UI and screen, which presents its own challenges given the constraints of a 2.4″ display. The UI is fully planned and prototyped, and button mapping is complete. While this is not yet connected to live data or the full system pipeline, we made the deliberate call to advance it as a parallel workstream to avoid blocking on system integration debugging.

Our demo objective is to showcase multiple subsystems working in isolation, demonstrating the breadth of what has been built. The critical success metric for the demo is achieving at least one complete end-to-end loop — a single full pass through the system from input to output. Outstanding work before the demo includes finalizing the prompting library and developing robust, dynamic suggestion generation, which remain active areas of development.

Following a meeting with our TA and professor, we finalized a testing plan that covers both dimensions of evaluation. On the technical side, system outputs will be graded against defined criteria. On the qualitative side, we will recruit a random selection of users to evaluate and grade system responses, giving us a human-centered signal on output quality alongside the technical metrics. This dual approach gives us a well-rounded framework for validating the system ahead of final evaluation.

Key decisions this week:

  • Shifted architecture from host-side to device-side execution
  • Adopted SQL query format for host-to-device data transfer to optimize GPU load
  • Started device UI development as a parallel track to unblock progress
  • Finalized dual-track testing plan with TA and professor sign-off

Current blockers:

  • Jetson and Pico communication stability
  • Full system integration and end-to-end data flow not yet complete
  • Prompting library incomplete; dynamic suggestion system still in development
Posted on

Sida’s Status Report for 3/28

  • My work: Spent significant time debugging and testing the full end-to-end loop: Host app → Pico → Jetson → LLM server → back to Host. Solved multiple issues to when bridging the app-pico connection with the pico-jetson connection, then made the full loop work. The results need tweaking, but the core functionality is completed and ready to demo
  • Major architecture refactor: Moved the pico firmware from QMK to Circuit Python. Due to the nature of the pico shifting from a keyboard to a transmission bridge between the host-side software and the LLM, the QMK firmware is capable but redundant. CircuitPython gives better development and debugging experience and is easier to work with. It still keeps the general HID functionality and can type back, acting as a keyboard if needed.
  • Quality of Life:
    • Implemented live capture deduplication so the polling feed only appends text when the content actually changes, avoiding redundant entries.
    • Added robustness improvements including automatic device reconnection after unplug/replug on both the Jetson bridge and host HID client, a self-window guard so SPARK doesn’t capture its own window, a fixed-size
      scrollable output window, and a quit button.
  • Pending work:
    • moving the host-side database implementation to Jetson
    • Review for the 2nd version of PCB design.
Posted on

Leonard’s Status Report for 3/28

This week, I assembled the initial version of the SPARK PCB and began integration. During this process, several issues were identified in the current design, including layout limitations that affected integration. Based on these findings, I reviewed the schematic and PCB layout and implemented necessary design updates. These revisions aim to ensure smoother integration in the next iteration of the hardware.

In parallel, significant progress was made on the system testing and verification plan. I developed a structured set of 20 testing scenarios, aligned with the four main SPARK features: context summary, keyword search, text formatting, and response drafting. Each feature is supported by five realistic, standardized scenarios designed to evaluate system performance based on a rubric of 6 categories.

For the upcoming week, my main focus will be on preparing for the interim demo and also waiting for the arrival of updated PCBs and replacement components.

Posted on

Tatyana’s Status report 3/21

This week I covered two main areas: completing the hardware-software HID bridge and beginning coordination on the Jetson output pipeline.

The primary technical completion this week was implementing and getting the keyboard communication module up and running on the Pico. This closes the hardware-software integration loop that was planned last week. The new module follows the same architectural pattern as the existing hotkey manager, running in a background thread, emitting signals on incoming packets, and wiring directly into the existing capture and release handlers in the host application. With the Pico confirmed working as a physical trigger, the keyboard can now initiate the full capture-process-release pipeline as a hardware input. This shifts the backend logic on to the firmware instead of having it completely on the software side. The communication,in byte format ,is also in place, allowing the host application to send status information back to the keyboard for LED or layer feedback.

The second area is a collaboration with Sida beginning this week on the receiving logic for the Jetson output. Once the Jetson generates a response, that output needs to be received on the host side, routed correctly through the pipeline, and delivered back to the user either displayed in the UI or pasted into the active application. As part of this collaboration the team is also evaluating whether to consolidate the keyboard firmware and the host application into a single unified repository rather than maintaining two separate ones. The goal is to reduce overhead and make the integration between the two sides of the system easier to manage as the project moves into its first demo.

Next week the priorities are investigating text extraction approaches for websites and Google Docs. The current accessibility based capture works well for native desktop applications but does not reliably expose content from browser based surfaces, so this will involve researching browser extension APIs, the Google Docs REST API, and other approaches to identify the most reliable and privacy consistent solution. The other priority is to continue work with Sida on the Jetson output-receiving logic to move toward a complete end-to-end run of the full pipeline.

Posted on

Team’s Status Report for 3/21

This week, the team’s focus is on integrating the connections between each part of the system and building toward a fully functional demo.

Between the host-side software and the Pi Pico, Sida and Tatyana have produced an MVP using the RAW HID reports. The protocol remains to be tested for heaviers load and actual feature calls, but the basic communication is tested to be functional.

Between the Pi Pico and the Jetson, Sida has produced an MVP using a UART connection between the GPIO of the Pi Pico and the 40-pin expansion header of the Jetson.

On the hardware side, most components have arrived, but the PCB has been delayed and is now expected next week. Once the PCB is in hand, Leonard can start with the physical assembly. In the meantime, Leonard is working on the testing suite.

The team also completed the ethical considerations discussion this week, including a worst-case scenario analysis and red-teaming feedback from other teams. We will incorporate the feedback into our interim demo and final showcase.

Next Week: The primary focus will be on hardware and system assembly to bring all parts together for a full demo. We will have separate meetings of two members for the integration tasks.

Biggest open risk: Both the PCB delay and delays caused by personal situations have some impact on the overall progress. We are 2-3 days behind our schedule, but we will make up for it to build the demo.