- Making the solution part of the Design Review presentation(system block diagram, implementation plan)
- Presented the Design Review presentation, waiting for the feedback
- Working on finalizing the BOM and starting on the firmware
Sida’s Status Report for 2/14
Tested multiple models that the Jetson AGX Xavier can run (all with llama.cpp and 4-bit quantization).
- Qwen3-4B: output 22 token/s
- Phi-3 Mini (3.8B): output 22 tokens/s
- Llama3.1-8B: output 15 token/s
- Nemotron-3-Nano, GLM 4.7 Flash cannot run due to the lack of memory.
Findings:
- Qwen3-4B and Phi-3 Mini are both able to perform correction tasks correctly and generate output at a similar speed. Llama3.1-8B is slower due to more parameters, and it could be a backup if the first two models cannot perform complex tasks later.
- Reading speed of the prompt is non-linear. For short prompts(<50 tokens), the reading speed is ~60 token/s –> ~0.3s. For long prompt(~500 tokens), the reading speed is ~300 token/s –> 1~2s. I need to test this further so we meet our design requirement.
- Theoretically, Nemotron-3-Nano and GLM 4.7 Flash should be able to fit into the 32GB memory, but they couldn’t unless being significantly tuned smaller, which perform slower and dumber than the above models.
Sida’s Status Report for 2/7
Team Effort(what I have done with other team members):
- Working on and presenting the proposal pitch
Individual Effort:
- Got ShearedLlama-2.7B(4-bit quantization) + NanoLLM + MLC working, but had to reduce the context length to 2048(maximum is 4096) in order to fit into the memory. Even then, it’s unstable and runs out of memory sometimes, and the performance is not very good.
- Acquired the Jetson AGX Xavier from the inventory, which has 32GB of memory, hopefully it will perform much better. Currently working on getting Qwen3 to work on the device.
Team Status Report for 2/7
After the faculty meeting on Wednesday, the biggest takeaways are:
- [scope ]need to prove that our solution/feature is interesting and valuable
- [technical feasibility] Uncertainty about whether the hardware can run the LLM well enough, and if the LLM can complete our tasks well enough.
So we
- Put together a user story that describes the scenario and pain points of users, and how our solution solves the problem and compares to other existing solutions. We defined our users to students managing multi-source academic writing, individuals with communication impairments needing typing assistance, and professionals with privacy concerns around intellectual property. These stories demonstrate how SPARK solves real problems and clearly differentiate our solution from existing alternatives like cloud-based AI assistants and manual workflow tools. We documented specific use cases including context-aware citation formatting, intelligent autocorrection that preserves meaning, and private workflow acceleration without data exposure risks.
- Working on deploying the most powerful model on Jetson Nano Orin, also waiting for more powerful devices that we can borrow.
Moving forward, we want to finalize our UI design with mockups of the touchscreen interface and physical button layouts, to make sure the flow supports our usability requirements. We’re also creating a very clear, step-by-step walkthrough of our product that shows the complete user experience from setup through daily use across different scenarios. This will serve as a way to keep us aligned later in development.
We are going to develop a testing plan that addresses both quantitative metrics like latency and acceptance rates, as well as the more qualitative parameters that faculty raised concerns about in our design presentation. Specifically, we need rigorous evaluation criteria for the correctness of grammar fixes, appropriateness of tone adjustments, and overall quality of generative content. Such as designing test cases with clear rubrics and planning what our user testing sessions would look like as well.
Challneges:
Time is our biggest blocker as we need to prove a lot that out lllm deployment is strong enough to accomplish what we want to do for our project.
Tatyana: User story development, testing plan framework
Sida: LLM deployment on Jetson,
Leonard: hardware research
Tatyana ‘s Status Report for 2/7
This week I created the agenda structure and organized group meetings. After our review the week prior I spearheaded an extra task to help us solidify our scope after critique around our use case and target users. I identified target groups to help shape what are needed mvp to address our target users. After creating this exercise, I focused on the context switch use case to help us understand the ideal use case as well as create a use case that would help us outline the limitations/ challenges.
After creating these user stories, we worked to create our proposal slide deck, which I presented. I was focused on the use case, key users, tech stack, and solution approach. As a result of presenting the deck, I created speaking points and prepared for the presentation.
In addition to the proposal, I also created the Gantt chart and timeline, as well as outlined team responsibilities/division of labour.
After our presentaion we worked to start defining UI after our presentation critique, create a better design for our device. Utilizing the hardware for finetuning/settings and pushing more of the user side interaction to software.