https://drive.google.com/file/d/1i8ojhM9djVCHblSq5gopb_jmhNufqRns/view?usp=sharing
Final Report
Final Video
Team Report 4/25
Unit Tests and Changes
This week we completed formal testing of the system against our quantitative design requirements. Our testing framework consisted of three components: predefined 20-word sentences for latency testing, paragraphs with labeled grammatical errors for accuracy validation, and repeated button-triggered actions across multiple trials to measure response consistency.
Across the four metrics we tracked, the system passed three cleanly. Autocomplete latency came in at a median of 0.661s and an average of 0.943s against a target of under 1 second. Grammar accuracy measured at 91%, clearing the 90% threshold. Context capacity came in at 8192 tokens, well above the 4000 token minimum. Full response time was the one inconsistent result, with a median of 1.861s but an average of 2.417s and a max of 7.161s against a 2 second target, indicating the system meets the requirement most of the time but has variability under certain conditions.
In addition to the quantitative testing, we conducted user testing sessions and collected verbal qualitative feedback from participants. That feedback was documented and summarized into a findings document that was handed off to inform refinement priorities going into this final week.
We decided to change the prompting style as well as the display of the information to help with the context/flow. Now the summary has more context and is more descriptive.
Based on testing findings, Sida updated the LCD UI to reflect a revised button layout to keep the on-device display in sync with the updated hardware interaction model. Additionally, the synthesis feature was reworked so it now pulls context from all relevant open applications rather than only the currently active window, which meaningfully improves the quality and completeness of the summaries the system can produce. This was a direct response to feedback about output relevance during user testing.
Risk
The primary remaining risk is the inconsistency in full response time, particularly the long tail at the high end. The team is aware of this and the focus going into the final week is on reducing that variance through continued prompting refinement and any remaining integration cleanup. No major changes to requirements or the block diagram were made this week.
Next Week
Going into the final week, the team’s focus is on polishing features based on user feedback, cleaning up the repository, completing the final poster and demo video, and consolidating all technical specs and lessons learned into the final report.
Tatyana Report for 4/25
This week my focus shifted toward testing support, qualitative research, and final presentation prep as the project moves into its last stretch.I helped source testers for our final presentation and assisted with troubleshooting some of the prompting and general refinement of the project. After conducting user testing, I collected a lot of verbal qualitative feedback from users and put together a document summarizing what we heard and what we should be improving going into this week. I handed that off to Sida, who used it to help improve the back end LLM prompting.
Since most of my front-end and technical work on the project is wrapped up, I shifted into more of a support role this week, helping Leonard with testing and taking on the layout and design of the final poster presentation.
Going into this week, my goals are to clean up the repository, work on the final demo and video strategy, and once the poster and demo are done, consolidate all of our lessons learned and final technical specs into the final report.
Sida’s Status Report for 4/25
- Updated the LCD UI to reflect the new button layout, ensuring the on-device display stays in sync with the revised hardware interaction model.
- Reworked the synthesis feature so that it now pulls context from relevant apps rather than only the current active app. This is a meaningful improvement to the core value of SPARK — the system can now produce more complete and useful summaries by considering the broader set of open context, not just whatever window happens to be in focus at the moment of the keypress.
- Next up: making the response drafting feature smarter, with improvements planned for next week.
Leonard’s Status Report for 4/25
This week, my primary focus was on completing the final presentation and refining the project content based on feedback received during the design review. I worked on improving the clarity and organization of the slides, particularly in areas related to testing, validation, and system functionality. Adjustments were made to better align the presentation with the project requirements, including clearer explanation of quantitative design requirements and more structured testing methodologies.
In the upcoming week, my focus will shift toward conducting additional testing based on the guidelines. This includes validating performance across the four main features and ensuring that results are consistent and measurable.
Team’s Status Report for 4/18
This week, the team reached its most significant milestone: full system integration is complete and all features are (somewhat) working end-to-end. Sida closed out the final firmware and software integration, Tatyana cleaned up the front-end and resolved a content scraping issue that had been degrading output quality, and the fixes came together into a system stable enough to put in front of real users.
The other major focus this week was user testing. The team designed and ran a structured user study, with standardized tester and moderator guidelines to ensure consistency across sessions. Results were analyzed and compiled into metrics for the final presentation, and the qualitative feedback from sessions will directly inform the polish work going into the final demo.
Next week: The team’s final push will focus on incorporating user feedback to refine features, doing more user testing, and preparing for the final demo.
Overall status: On schedule. Integration is done, testing has begun, and the remaining work is refinement.
Tatyana Status Report 4/18
This week was mostly a collaborative week as the team worked together to bring the project into its final MVP form ahead of the demo.
On the application side, I finished up the front-end application and got it into its final MVP state. This included removing leftover testing buttons, cleaning up the UI, and integrating a new content scraping library that finally works to strip out headers and unnecessary information that had been getting passed through the pipeline. That extra noise was degrading the quality of our output, so getting clean text extraction working was a meaningful step forward.
Most of my week was focused on testing. I worked with Leonard to create and refine the testing materials, where I took on building the rubric and designing the general testing flow. I created the forms, conducted user tests, and also helped troubleshoot some final integration issues we were running into on Mac.
After testing, I helped with the final presentation by creating the metrics slides. This involved doing some Excel analysis to bucket the form criteria into a cohesive metrics table. I also gathered qualitative insights from the user sessions that will help us going into next week as we look to refine features and improve the MVP for the final demo.
Going into our final week, the goal is to use the user feedback to polish features and make the project as strong as possible for the demo.
Leonard’s Status Report for 4/18
This week, my primary contribution focused on developing the tester and moderator test guidelines, as well as creating a tester categorization form to improve the structure and usefulness of user testing. The tester guidelines were designed to standardize how participants interact with the system, ensuring consistency across trials. The moderator guidelines provide clear instructions for facilitating tests and recording observations. Additionally, the tester categorization form introduces a way to group participants (e.g., writing style, response length preference), allowing for more meaningful analysis of testing results.
In parallel, I also finalized the presentation slides for the upcoming Final Presentation, which will be the main focus next week. The plan is to make adjustments according to the comments in the presentation to prepare for the Final demo.
One key thing I learned from the project is designing standardized testing methodologies, including how to create controlled scenarios and measurable evaluation criteria. This involved understanding how to reduce variability in user behavior while still reflecting realistic use cases. Additionally, I learned how to structure user study documentation, including moderator scripts and participant instructions, which are critical for consistent data collection.
Another important knowledge is human-centered evaluation, specifically how to categorize users based on behavioral patterns (such as writing length, response style, and interaction habits) to extract more insightful results from testing.
Sida’s Status Report for 4/18
- Completed final integration of the full SPARK system — all features are working end-to-end and the system is ready for testing and demo.
- Decluttered and reorganized the repository to significantly lower the barrier to getting started. This includes one-click setup scripts that automatically configure the SPARK app on a fresh Windows or macOS environment, removing manual dependency setup steps.
- Helped prepare materials and testing results for the final presentation.
What new tools or knowledge did you need to learn?
Repo Management: Clearly defining the file structure of the repo early on can significantly boost clarity, productivity, and readability. Things will get messy really quickly, especially when collaborating with others. Same with Git best practices: do commit and commit often, branch management, commit message style, etc.
Development Environment Management: same ideology. What may work on your machine may not work on your other machine, let alone your teammate’s one. This is especially true for our team: I work mostly in Windows, and Tatyana works mostly in macOS. Carefully selecting mutual toolchains and navigating the OS-specific tools is 100% needed. Sometimes, hardcoding is necessary; most of the time, it should be avoided.
Agile Development for PCB: for early prototypes(which is every capstone project – no project would REALLY pass the early prototype stage), have more flexibility. Have test points, backup vias and pads for manual soldering and rerouting, 3d modeling for spacing, mounting holes for casing. order extra components… All very necessary. When making iterations, one should try to use as much already-bought components as possible.
The learning strategy was mostly hands-on and informal: running the scripts repeatedly on clean environments to catch failures, reading platform-specific documentation and Stack Overflow threads for edge cases, and iterating quickly based on what broke. This trial-and-error approach on real machines was far more effective than any top-down reading would have been for a task this integration-heavy.