- Finished the Design Review presentation together.
- Met and worked with TA to further refine our use cases and user stories
- Specifically, we pivoted the idea from a writing assistant to a more powerful workflow assistant, powered by the more advanced hardware(Jetson AGX Xavier) we were able to borrow from the ECE Inventory.
- The workflow assistant keeps track of the state of the system to perform four core functionality:
- Quick Summarize: Summarize what the user has been working on for the last 20-minute session. This can quickly provide context to chatbots, co-workers, or personal jouarnal.
- Keyword Search: Search deeper into more distant records of inputs and context to quickly get back on track of last changes.
- Auto Edit: Based on the context, edit the previously typed sentence to be grammatically correct and matching in tone.
- Auto-completion suggestions: Based on previous context, generate smarter auto-completions with local LLM.
- Started with the template for the Design Review Document
Sida’s Status Report for 2/21
- Making the solution part of the Design Review presentation(system block diagram, implementation plan)
- Presented the Design Review presentation, waiting for the feedback
- Working on finalizing the BOM and starting on the firmware
Tatyana Status Report 2/21
This past week, the SPARK project moved from the drawing board into a functional software prototype. My primary focus was building the side panel that sits on the desktop. I successfully developed a custom user interface that stays on top of other windows.
The most significant shift this week was moving away from complex modes and committing to four general actions: Synthesis, Content Generation, Recall, and Short Response. This helped us strengthen our scope by narrowing it.
On the technical side, I implemented a “Context Tracker” that monitors window activity every half-second, logging which applications are being used and scraping relevant text snippets into a local database. To verify that the interface and database are communicating correctly, I also built the infrastructure for the four action buttons. Next week, I aim to focus on creating basic logic responses and starting the llm integration.
One challenge I encountered involved hardware limitations on my MacBook. With only 8GB’s of ram, running large AI models locally causes the system to slow down significantly. To address this, I developed a scaling strategy , starting with a smaller, highly efficient model called Phi-3-mini to build out the LLM pipeline before moving to larger models.
In addition to development work, I contributed to our design documentation, specifically the introduction and use case sections.
Leonard’s Status Report for 2/21
This week, I completed the first draft of the circuit schematic for the SPARK hardware system. The schematic includes: RP2040-based MCU, SPI display, and function buttons.
For future weeks, the plan is to order the parts and assemble them on a breadboard first. Once the system is validated, the circuit will be designed on PCB.
Team Status Report for 2/14
This week, after the meeting on Wednesday with faculty and TA, the team thought that defining the right use-case example and requirement are still the main issue.
The team decided to refine SPARK’s core use-case requirements, which include four primary action categories: Synthesis, Text Reformatting, Keyword/Phrase Searching, and Response Drafting. Each use case was rewritten to include measurable behavior and concrete examples to better define expected functionality.
The team also worked together on making the block diagram for the SPARK system and discussed and prepared the design review slides for the presentation next week.
=======================================================
Part A (Sida) with respect to considerations of public health, safety or welfare:
Regarding public health, some people have developed a psychological attachment/affinity to their chatbots. Although our project’s features don’t include chatting with the LLM, we still need to be cautious about the longevity of our project. Our usage of locally hosted and open-source LLMs ensures that users won’t suddenly lose access to the model.
Part B (Leonard) with consideration of social factors:
On the topic of social factors, there are many concerns about AI replacing the human workforce. Our project’s scope is to increase the user’s work efficiency, not to replace any human effort. It remains to be determined what AI’s impact on society will be, but our goal is to empower each user.
Part C (Tatyana) with consideration of economic factors:
The cost of using AI poses an economic barrier to people with different financial statuses. Large businesses can afford to pay hundreds of thousands of dollars to self-host SOTA models or use API services. Individuals can pay monthly subscriptions to use less-capable models at slower rates. Self-hosting a decent model for everyday tasks isn’t financially reasonable yet, but our project can show that smaller models can be used to perform specific tasks and can meaningfully empower users with a one-time investment.
Sida’s Status Report for 2/14
Tested multiple models that the Jetson AGX Xavier can run (all with llama.cpp and 4-bit quantization).
- Qwen3-4B: output 22 token/s
- Phi-3 Mini (3.8B): output 22 tokens/s
- Llama3.1-8B: output 15 token/s
- Nemotron-3-Nano, GLM 4.7 Flash cannot run due to the lack of memory.
Findings:
- Qwen3-4B and Phi-3 Mini are both able to perform correction tasks correctly and generate output at a similar speed. Llama3.1-8B is slower due to more parameters, and it could be a backup if the first two models cannot perform complex tasks later.
- Reading speed of the prompt is non-linear. For short prompts(<50 tokens), the reading speed is ~60 token/s –> ~0.3s. For long prompt(~500 tokens), the reading speed is ~300 token/s –> 1~2s. I need to test this further so we meet our design requirement.
- Theoretically, Nemotron-3-Nano and GLM 4.7 Flash should be able to fit into the 32GB memory, but they couldn’t unless being significantly tuned smaller, which perform slower and dumber than the above models.
Leonard’s Status Report for 2/14
This week I focused on the hardware architecture for SPARK and finalized several key component decisions, including the Raspberry Pi Pico (RP2040) as the MCU and the Waveshare 2.4” SPI LCD module as the display.
For user input, there are four pushbuttons for core interactions, a three-position slide switch for different mode selections, and a rotary encoder option for future UI expansions.
For future weeks, I plan to assemble the full system on a breadboard to verify functionality and stability. The goal is to validate hardware behavior before committing to schematic capture and custom PCB layout design.
Tatyana Status Report 2/14
This past week, I focused on solidifying use cases, prototyping the software user-side workflow, and preparing the design presentation.
Following feedback on our use case, I integrated the suggestions to ensure our use cases showcase the most powerful aspects of our project. As a result, I created use cases and walkthroughs for the four core functionalities, enabling us to give a detailed walkthrough of our product as if it were demo day. By narrowing the project scope to improving the student workflow, we were able to focus on the key areas of the student workflow experience that can be enhanced with the SPARK device.
I developed these cases by working on the database design and building a prototype of the context-capturing application. On the software side, I implemented six core features: global text capture to grab selected text from any application, window tracking that monitors the active app, browser tab, or window every two seconds, and context awareness that extracts text from focused elements to build a running history of the user’s workflow. I also implemented persistent storage to save window context snapshots to a database, smart flushing that aggregates and distills captured context after 50 window switches for LLM processing, and text paste-back using Cmd+Ctrl+R to release processed text back to the active application. Together, these features allowed me to explore how much context could be captured and how the workflow could be linearized in the database.
After making progress on my technical deliverables, I worked on the design presentation, focusing on use cases, solution approach, and project management materials. In addition, I created the first draft of our system block diagram.
Next week, I want to work on implementing the groundwork for the data capturing as well as the clean up and delivery of json data, to prepare for LLM integration.
Database
SPARK window
Leonard’s Status Report for 2/7
This week, I focused on defining the testing, verification, and metrics framework for SPARK. I helped formalize how we will verify core design requirements and validate use-case requirements. For each requirement, there is at least one concrete test and associated quantitative metric. This will provides a clear basis for evaluating performance beyond a simple functional demo.
In addition, I began planning a standardized testing sample for SPARK. The goal is to create a consistent set of writing tasks so that feedback from different users is comparable and not overly influenced by individual writing styles or habits. This will allow us to collect more meaningful and repeatable evaluation data.
Sida’s Status Report for 2/7
Team Effort(what I have done with other team members):
- Working on and presenting the proposal pitch
Individual Effort:
- Got ShearedLlama-2.7B(4-bit quantization) + NanoLLM + MLC working, but had to reduce the context length to 2048(maximum is 4096) in order to fit into the memory. Even then, it’s unstable and runs out of memory sometimes, and the performance is not very good.
- Acquired the Jetson AGX Xavier from the inventory, which has 32GB of memory, hopefully it will perform much better. Currently working on getting Qwen3 to work on the device.