Posted on

Charlie’s Status Report for 4/25

This week, I focused on revising the server architecture and refactoring the state bridge to address limitations from the previous single-channel WebSocket model between the web client and Rust engine. While the earlier design worked for the React frontend, integrating additional hardware components such as CV-based board state detection and LED control required a more robust distributed architecture to support asymmetric clients. The updated design establishes the bridge as the single orchestrator and external gateway, with the engine reduced to pure game logic and stateless analysis. All components—including UI, CV, LED, and coaching—now interact through a unified bridge-owned command/event model, ensuring consistency and eliminating split-brain state. This redesign also ensures that all authoritative moves are broadcast as events, prevents helper operations from mutating live state, and enables CV-detected moves to act as first-class command inputs. I also revised the LED light configuration on the client to better match the physical board design, improving visual alignment between digital and hardware feedback.

In parallel, I continued developing the agent orchestration system, implementing a fast-path execution flow and iterating on decision routing based on board state. I integrated the Go-based coaching service into the new architecture and expanded integration testing across the bridge, client, and agent layers. I also conducted additional testing on the RAG pipeline, fixing issues with context injection and improving prompt constraints for token usage. . For better agent observability, I added a visualization and logging panel to display tool usage and track all agent actions in real time. On the data side, I collected more game position samples for puzzle generation and began revising the puzzle curator agent and state detection logic.

Overall, my progress is mostly on track, with continued focus on stabilizing the new architecture and improving system reliability.

Posted on

Charlie’s Status Report for 4/18

This week, I rewrote the agent orchestration framework in Go, introducing structured agent state management along with defined skills and tools integrated with the engine. This significantly improved observability and debugging, as the system now uses Prometheus and Grafana along with a dynamic React dashboard to visualize agent execution, parallel node activity, and tool usage in real time. In addition, to improve user interaction beyond speech-to-text input, I developed a virtual avatar using Three.js that maps engine evaluations to animations, providing more intuitive and engaging feedback to the user during gameplay.

During development, I identified limitations in the current embedding strategy for fine-tuning, particularly with Qwen 7B, where dense vectors made it difficult to align outputs with evaluation signals. As a result, I pivoted toward a heuristic-first approach for move explanation generation, incorporating factors such as game phase, pawn structure, center control, and king safety. I also implemented a LoRA fine-tuning pipeline using expert commentary data, but am delaying full training until the heuristic mapping layer is complete.

Throughout this capstone, I learned several new tools and concepts, including agentic frameworks, observability tools like Prometheus and Grafana, and LoRA fine-tuning techniques. I primarily relied on informal learning strategies such as reading Medium articles, technical blogs, and documentation to quickly get up to speed and apply these tools effectively.

Posted on

Charlie’s Status Report for 4/4

This week, I continued working on the LLM agent orchestration pipeline and prompt engineering, focusing on improving reasoning quality and reducing hallucinations in move explanations. I also explored deployment options using Amazon Bedrock and researched LoRA fine-tuning as a potential approach for adapting models to the Xiangqi domain. Through this investigation, I identified a key limitation in standard LLMs: lack of spatial awareness when interpreting raw board representations such as FEN. To address this, I designed a feature engineering approach that converts board states into a structured relational representation, encoding piece interactions, threat maps, palace dynamics, and pawn structures. This effectively transforms the problem from spatial reasoning into a relational reasoning task, which aligns better with transformer-based models.

Additionally, I began prototyping this relational preprocessing pipeline and integrating it into the agent workflow alongside RAG. The goal is to provide the LLM with structured, verifiable context to improve consistency and interpretability of generated explanations. This approach reduces reliance on implicit spatial reasoning and instead supplies explicit constraints derived from the engine, helping mitigate hallucinations. Moving forward, I plan to refine the feature extraction process, validate its impact on explanation accuracy, and continue improving prompt design and integration with the full system pipeline.

I plan to evaluate the LLM orchestration system using  multiple metrics and  structured logging and agent visualization. First, I will measure explanation correctness by checking whether the LLM’s suggested moves and reasoning align with the engine’s validated outputs. I will also track hallucination rate, calculated from the percentage of responses that include invalid moves, incorrect checkmate claims, or inconsistencies with the game state. I will also evaluate consistency by running repeated prompts on the same position and measuring differences in outputs. For performance, time between text and move input and when response is returned  is measured  to ensure we meet the real-time feedback latency. For debugging purpose, I will implement detailed agent logging of intermediate steps, including retrieved context, tool usage, and sub-agent calls, and use this data to compute metrics such as tool-call accuracy and pipeline success rate . Visualization tools will complement this by mapping explanations to board states and candidate moves, enabling faster debugging, but evaluation will remain grounded in measurable benchmarks.

Posted on

Charlie’s Report for 3/28

This week, I focused on improving the engine’s move generation performance by applying additional optimization techniques such as quiescence search (Q-search) and transposition table lookups (TTLookup). These changes significantly reduced the effective branching factor and improved overall search efficiency, especially when combined with move ordering strategies. In parallel, I continued developing the agent orchestration pipeline by incorporating retrieval-augmented generation (RAG) and experimenting with prompt designs for more accurate and coherent explanation generation.

I also integrated the full pipeline with the frontend, enabling end-to-end interaction between the UI, engine, and explanation system. Additional integration tests were added to validate system behavior and ensure consistency across components. Moving forward, I will continue refining prompt engineering to improve explanation quality and further optimize the interaction between the search engine outputs and the LLM-generated reasoning.

Posted on

Charlie’s Report for 3/21

This week, I continued working on the LLM agent orchestration pipeline, focusing on structuring how game state, candidate moves, and contextual information are passed into the system for explanation generation. I explored approaches for maintaining consistent agent state and improving how retrieved context is incorporated into the reasoning process, with the goal of reducing hallucinations and ensuring alignment with the engine’s validated outputs.

I was not able to conduct user testing for the UI this week as originally planned. As a result, I am slightly behind on the evaluation of the interface and interaction design. Next week, I plan to prioritize user testing to gather feedback on usability and make necessary adjustments, while continuing to refine the agent orchestration pipeline.

Posted on

Team’s Status Report for 3/14

The most significant risk that could jeopardize the success of the project remains the board state detection accuracy, which has not changed since last week. Our goal is to achieve above 95% accuracy in detecting board states from the physical board. However, even at 95% accuracy, this would correspond to approximately one incorrect detection every 20 moves, and with typical games lasting around 40–60 moves, this could introduce multiple errors during gameplay. Such misreads could propagate incorrect game states to the engine and affect move recommendations. To mitigate this risk, we plan to develop more robust evaluation metrics to identify specific failure cases, such as misclassification of certain pieces or errors under particular lighting or board conditions. As we continue testing with the physical board and pieces, we will focus on improving detection reliability and identifying conditions that cause failures.

We do not have any changes to the existing design requirements and have no updates to the project schedule this week.

For individual status reports, see:

Charlie – http://course.ece.cmu.edu/~ece500/projects/s26-teamd3/2026/03/15/charlies-progress-report-for-3-14/

Yoyo – http://course.ece.cmu.edu/~ece500/projects/s26-teamd3/2026/03/14/yoyos-status-report-for-3-14/

 

 

Posted on

Charlie’s Progress Report for 3/14

This week, I finished building the frontend user interface for the chess application and integrated it with the existing backend components. The interface now supports visualization of the board state and user interactions with the system, allowing moves and engine responses to be displayed in real time. In addition, I began working on the LLM agent state management and retrieval-augmented generation pipeline, researching how to structure context and maintain consistent reasoning for move explanations.

My progress this week is generally on schedule. Next week, I plan to conduct user testing to evaluate the usability of the UI and gather feedback on interaction design. I will also continue developing the agent state management and RAG components to improve how the system generates and maintains contextual explanations for gameplay decisions.

Posted on

Charlie’s status report for 3/7

This week, I focused on building the frontend user interface using React and integrating it with the backend systems. I implemented core UI components for interacting with the chessboard and began setting up integration tests to verify correct communication between the frontend and backend. These tests help ensure that board state updates, move validation, and engine responses are properly synchronized across the system.

In addition to development work, I conducted further research into implementing retrieval-augmented generation (RAG) and agent state management using LangChain. This exploration focuses on how the system can maintain contextual knowledge and structured reasoning when generating explanations for moves. The goal is to design a pipeline where the engine outputs and relevant game context can be retrieved and incorporated into the LLM’s reasoning process while maintaining consistency with the validated move set.

Posted on

Charlie’s Status Report for 2/21

This week, I have completed our Design Presentation and begun to incorporate feedback into our system architecture. On the implementation front, I have completed the rule validation layer and full game state tracking, including turn management and legal move constraints. I have also started working on the move generation system, organizing it in a way that allows it to cleanly interface with the existing validation system and future search algorithms.

Concurrently, I have begun working on unit tests to ensure correctness and started to benchmark the current engine performance. I started working on the user interface for the chessboard to enable visualization and interaction with the engine. Because of the extra time needed for proper testing and integration, I am currently a little behind schedule.

Posted on

Charlie’s Status Report for 2/14

This week, I focused on preparing the Design Presentation and refining the system’s user interaction model. In addition to slide development and updating system diagrams, I designed and began implementing the user interface for the chessboard, including real-time board state visualization synchronized with the engine. A key design change was shifting from a primarily voice-controlled interaction model to an interface that includes explicit on-screen controls alongside the live camera feed and board state display. This adjustment improves usability, reduces ambiguity in user input, and provides more reliable interaction during testing and demonstrations while still supporting guided feedback from the engine.