Category: Nicole’s Status Reports

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

• This week I enhanced the web app with several new features. I integrated and debugged all these features so that they work with the hardware on the RPi.
  • I enhanced the “missing block” feature to report the position of the blocks that are missing or have a bad connection.
  • I added sound effects and color changes for the submission checking pages to draw the user’s attention to the web app screen. In user testing, we noticed that users would miss the answer-checking result screen because they would be too focused on the blocks and not looking at the web app.
  • I implemented and integrated an “easy” mode which is 8 blocks and 2 categories, as opposed to the regular 16 blocks and 4 categories.
  • I implemented and integrated a feature that lets us play a puzzle given a specific game ID, which we can use to test different puzzles during user testing.
  • I added a “super hints” feature where you can see 1 word in each category (chosen randomly).
  • I fixed some frontend bugs where the mistake count would disappear in certain cases.
  • I added a “game status” check on each page that checks the number of categories found and mistakes remaining and transitions to the lose/win screen if needed. This should minimize the chance that we somehow get stuck in a state where all categories have been found but the web app doesn’t transition to the win page, or the situation where the mistake count goes negative and still allows the user to play.
  • I created a database (giant json file) of pre-retrieved hints for each word (~5k) generated from Gemini. This way we can cut down on the hint retrieval time since we can just fetch them from the database instead of generating them in real time.
• I started researching using neopixel strips with the on-blocks picos to enhance the aesthetics of the game.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

• My progress is on schedule. I’ve pretty much completed all my tasks and will just be focused on helping my teammates with enhancing the aesthetics/cosmetics of the project. 

What deliverables do you hope to complete in the next week?

• Next week I will complete the video and final report. I also hope to conduct 5 more user tests with our newly enhanced web app features. If time permits, I hope to get neopixels working on the blocks as well.

 

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

• This week I completed our hint retrieval latency and accuracy test. I tested our entire database of puzzles, gathering timing and performance accuracy metrics with regards to the hint retrieval. I recorded how many words from each puzzle did not have any definitions nor context retrieved by the merriam webster dictionary API, and I timed how long each API call took. I then analyzed this data to gather statistical performance metrics in terms of average latency of hint retrieval and the quality of the retrieval. 
• I found that there were puzzles that have unretrieved hints for multiple words, which is unideal for the use case of our project, so supplemented our dictionary API hint retrieval with hints generated from an LLM (in our case, gemini flash-2.0 model). I then tested the entire database again with this new capability and did a quality/latency trade-off study on the different LLM models available and different mixtures of APIs.
• I helped to construct the final grid. I helped come up with a solution to our pogo pin alignment problem and also helped in physically soldering connections on the circuit boards and other wired components on the grid. I then helped to debug the final circuits as there were some accidental shorts. 
• I rewrote the web app to use SocketIO in place of the POST requests I was mainly using in our previous implementation. Previously, I was having trouble with getting the sockets to work consistently on the RPi as it seemed some signals were just never received. I had implemented a work-around using the return status codes of POST requests instead of socket signals, but after testing out our final grid implementation and seeing that we still suffered from alignment/connection issues, we decided that we would need to implement an error message feature that would require the use of sockets for one-way server to client communication. I was able to get a flask-socketio implementation to work on my mac laptop using a greenlet (eventlet) framework (I figured out that the issue I was facing before was due to the synchronous nature of the socketio function calls, but when it came time to integrate it onto the RPi, I ran into issues.
• It turns out that eventlet/gevent (greenlet frameworks) are incompatible with the RPi.GPIO library since they require monkey-patching in order to enable asynchronous socket capabilities. I needed monkey-patching to get the sockets to work, but it turns out this implementation is not possible to run on the RPi. Instead of eventlet, I researched other alternatives and ultimately found that uvicorn, an ASGI web server, along with python’s native asyncio package should work with RPi GPIO. However, this package does not have Flask support, so I rewrote the entire Flask backend as a FastAPI backend instead. I then translated the appropriate frontend socket logic as needed, then reintegrated with the RPi and found that it worked great.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

• My progress is a little behind schedule because we were having lots of issues during integration for the final product. As a result, I didn’t get to do as much user testing this week as I wanted to. However, now everything (minus aesthetics) is basically done and ready for user testing, so I will block out time next week to finish the user testing that I didn’t get to do this week.

What deliverables do you hope to complete in the next week?

• Next week I hope to complete our user testing. I also want to get started on the poster and final report. Additionally, I will work on making our project more aesthetic (think about painting/branding).
• I also want to implement a feature on the web app where it displays the position of the block that is missing instead of just generically saying that a block is missing.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

We recognize that there are quite a few different methods (i.e. learning strategies) for gaining new knowledge — one doesn’t always need to take a class, or read a textbook to learn something new. Informal methods, such as watching an online video or reading a forum post are quite appropriate learning strategies for the acquisition of new knowledge.

• I had to learn about backend frameworks such as Flask and FastAPI, CORS policies, as well as using web sockets in python. To learn about these frameworks, I watched many tutorial videos on building React + Flask/FastAPI applications, and read deep into the documentation about the SocketIO/Flask-SocketIO and asyncio packages for python. Additionally, I had to read many stack exchange and discussion forum posts to diagnose specific issues I was having with integrating the web app on the RPi and with CORS issues.
• I also had to learn how to use the merriam webster dictionary API and particularly how to parse the json response from it. In order to do this, I read the github pages of other authors who created wrappers for the merriam webster API to get inspiration.
• Finally, I had to learn about using generative LLM APIs such as gemini. To do this, I read deep into the documentation and read through their provided example programs.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

• This week I fabricated and tested all the internal circuitry for the new 8 blocks.
  • I soldered all battery holders to adapters, soldered headers onto all picos and muxes, cut charging ports into battery holders, and tested all picos, waveshare LCDs, batteries, battery holders, and adapters for functionality.
• I changed the web app to handle 4 rows by using the full NYT Connections dataset and using their startup word placements instead of a random shuffle. I also implemented the “one word away” feature.
• I helped to fabricate the test bed for 16 blocks. I laser cut cardboard platforms for the male pogo pins, cut all the wires needed, and helped to tape everything onto a text board.
• Additionally, I soldered wires for the buttons and row LCD displays, and did a little soldering to help with the transfer of using a breadboard to using a soldered perf board for the grid circuitry.
• I went through the NYT connections database and picked 2 puzzles of similar difficulty that contained words involving knowledge of American cultural references and double meanings to use for our user testing.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

• My progress is a little behind schedule because we were having some issues with the 16-block grid working consistently, so I did not get as much testing as I would have liked. To remedy this, I will write scripts to perform my latency testing so that once the grid is ready, I can just run my scripts and get the testing done as fast as possible.

What deliverables do you hope to complete in the next week?

• Next week I hope to complete my verification tests and gather user testing data.
• If I have time, I will also try to improve the text cleaning for the dictionary api and potentially look into using gemini api for words or phrases that don’t show up in the dictionary api. 

Now that you have some portions of your project built, and entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements? Verification is usually related to your own subsystem and is likely to be discussed in your individual reports. Validation is usually related to your overall project and is likely to be discussed in your team reports.

Tests that I have run:

• Game logic accuracy
  • Input: Sets of submitted words
  • Output: 100% correct answer checking
  • I played 10 games. For 5 games, I tested 2 combinations of correct submissions and 4 combinations of wrong submissions. I made sure that 4 wrong submissions resulted in a game loss. For the next 5 games, I tested 2 combinations of wrong submissions and 4 combinations of correct submissions. I made sure that the 4 correct submissions resulted in a game loss. For all games, I made sure that the correct submissions were registered as correct and the wrong submissions were registered as incorrect. I also verified that the mistake count was decremented and reset properly, and that submitting the same category multiple times would not result in a premature game win. For each of the 10 games, I made sure that at least 1 wrong submission was a “one away” submission, and checked that the “one away” message was displayed properly.
  • From these tests, I was able to verify that the game logic was 100% accurate, which is imperative as incorrect game logic will result in the user not being able to finish the game properly since each successive submission depends on what words have been “eliminated” by previous submissions.

Tests I am planning to run:

• Hints Retrieval Latency
  • Input: Load 20 puzzles
  • Output:3.2s latency/puzzle
  • I plan to write a script that will run on the RPi to test latency of hint retrieval. The script will load 20 puzzles. For each puzzle, I will time how long it takes to retrieve/generate the puzzle, retrieve hints via calls to the dictionary API, and parse and store these responses into the game controller data structures. This will give us an idea of how long it takes to gather hints for all of the words in a puzzle, and make sure that it is not too long so that we can meet our latency goals and maintain user attention.
• Hints Retrieval Quality
  • Input: all words in the NYT Connections Archive dataset
  • Output: 
    • 95% retrieval rate of definitions
    • no more than 1 word’s hints unretrieved for each puzzle
  • Aside from the latency, I also want to test the quality of the hint retrieval itself. I will write a script to loop through every word in the dataset and record the number of times where no definitions or context/examples were found for a word. I want to make sure that definitions/examples are retrieved at least 95% of the time. Additionally, for each puzzle, I want to make sure that no more than 1 word in that puzzle does not have hints available. If I do not meet this goal, I will consider options like generative LLM APIs such as Gemini to provide hints for missing words. It is important that the virtual companion is able to retrieve hints reliably since otherwise it will be completely useless to the user as a feature.

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
  • This week I successfully implemented my workaround for the SocketIO bug. My new solution was to just use the return status of the POST request from the frontend to trigger the page transition. This logic is cleaner, more simple, and less prone to failure.
  • I also helped to re-fabricate the blocks for testing after Alexis cut holes for the on/off switch.
  • Additionally, I have been testing the entire game flow E2E with the hardware and software (demo), and noting down bugs/errors that I find, so that we can continue with fine-tuning our integration of the hardware and software components.
  • I also fixed an edge case with the dictionary where if a word doesn’t appear in the dictionary, it causes a network error during puzzle uploading to the blocks.
• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • My progress is on schedule since we are pretty much done with our MVP (minus the grid construction). Now I just have to finish up the testing to see if we meet our design and user requirement specs.
• What deliverables do you hope to complete in the next week?
  • Next week, I hope to start fabricating the next 8 blocks that we will need for our reach goal of 16 blocks. I’ll be soldering all the necessary parts together to make the internal circuit for the blocks. I will also try to finish up my testing on our MVP, and hopefully be able to help with grid design as well.

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
  • This week I prepared wires to solder onto the buttons and soldered the remaining power adaptors onto the remaining battery holders. I also helped test each of the pogo pins after soldering to make sure they had solid connections (doing a simple power test using a power source and a DMM to measure voltage). Additionally, I helped to build the circuitry for the grid and organize the connections from the RPi to the rest of the components.
  • I also helped Wen Hui in integrating the embedded controller with the web app, and was able to get uploading words to the blocks and button submission (link) for both rows to work, all through the web app.
  • After trying multiple methods to try and fix the SocketIO issue between the web app backend and frontend, I still was not able to resolve the issue. However, I came up with a new contingency plan or solution wherein I just make a button on the frontend appear after the post request to the backend returns successfully, so the user can manually transition to the gameplay page. This works because the problem right now is that the backend function returns successfully, so we are ready to start the new game, but the frontend never receives the socket signals that make it transition to the next screen, so the user is stuck on the loading screen unless they know the url of the next page and manually go there themselves. By adding a button, the user can start gameplay manually when the set stage is successfully completed.
• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • My progress is a bit behind where I’d like to be due to the bug I was unable to fix. However, now that I have a new workaround idea that should be fairly straightforward to implement, I should be able to get back on track with integration testing and web app quality testing on the RPi. 
• What deliverables do you hope to complete in the next week?
  • Next week, I hope to implement my new workaround for the web app SocketIO bug that I was unable to solve. I also plan to continue integration testing to make sure that the hardware can reliably work with the web app. Additionally, I want to add some instrumentation to the code so that we can see how much time each stage is taking, and assess whether we are meeting our design and user requirements.

 

 

 

 

 

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

• This week I worked on trying to Dockerize the web app so that I could easily run it on the RPi. I was able to get the docker to run on my personal laptop, but dockerizing the web app seemed to make a specific non-deterministic bug become more apparent related to socket signals being sent and received between the frontend and backend. I was able to get the web app up and running locally on the RPi by installing all the packages and libraries directly onto the RPi, but I hope to get the docker working so that the environment can be more stable. I made some frontend changes to the web app on the RPi to adapt to the RPi’s screen size in the browser.
• I identified a non-deterministic bug relating to socket signals being sent between the frontend and backend and am currently trying to diagnose the cause of this fault, as well as possible solutions, which may require re-implementing the communication between the frontend and backend.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule.

• My progress is a bit behind where I’d like to be due to the bug I found. I had to stop my testing due to this bug. I am currently in the middle of debugging and am trying a few different approaches to make the communication between the frontend and backend more reliable. I will try to fix this bug as soon as possible so that I can continue testing and get back on schedule. However, I should still be able to help Alexis and Wen Hui with integrating the web app with the embedded controller since this specific bug is not necessarily a blocker.

What deliverables do you hope to complete in the next week?

• Next week, I hope to fix the bug I found and then resume the testing and validation of the web app on the RPi. I suspect that once this bug is fixed, the docker will also work. In that case, if time permits, I will work on getting the docker to run on the RPi.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

• • This week I worked with Wen Hui and Alexis to hash out the interfaces and handshaking signals between the embedded controller, game controller, and the web app. I wrote out the requirements for these functions in detail in this file here.
  • I cut out ports into the battery holders so that the charging ports were accessible.

    • 

  • I helped Alexis make laser-cut files for the bottom and side of the blocks, using measurements from our physical components.
    • 
  • I finished integrating the Merriam-Webster API into the web app. I wrote several functions to help make an API call and then parse the request to extract the specific information I needed from the complicated API response. The function I wrote can be found in this file here.
  • Most of my time this week was spent on writing my portions of the design report, formatting it, and cleaning up the diagrams for it. 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

• • My progress is mostly on schedule. I tried to set up the web app on the RPi this week, but ended up deciding to switch to packaging it all as a Docker and then just running that on the RPi instead so that I wouldn’t have to worry about environment dependencies. In terms of testing, I manually tested the web app game logic and hint retrieval latency/robustness through many games, but have yet to do it quantitatively according to our written out testing plan as I wanted to do this once the web app was successfully running on the RPi.

What deliverables do you hope to complete in the next week?

• • Next week, I hope to be able to host the web app on the RPi 5 through Docker. Once hosted on the RPi, I hope to start our testing plans for the game logic accuracy and hint retrieval latency. I also plan to help with the mass-fabrication of the blocks.

• What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
  • This week I finished implementing the web app front end pages and the entire backend for the web app. I created all the backend API functions that the game/embedded controller would just need to call in order to control the game flow display on the web app. I was able to simulate a game session by making POST requests to my web app backend.
    • link to Flask backend file: app.py
  • I registered for the Merriam-Webster API and obtained API keys.
  • I also helped Alexis to prototype UART communication between the RPi and the pico on a breadboard using the pogo pins.

• Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
  • My progress is mostly on schedule. Currently the web app is still using the free dictionary API, since I still have to figure out how to parse the Merriam-Webster dictionary API output to extract the definition and context for a word. I need to research a bit into the documentation for the Merriam-Webster dictionary API to see how I can do this.

• What deliverables do you hope to complete in the next week?
  • Next week, I hope to have the Merriam-Webster dictionary API fully implemented into the web app, and I hope to be able to host the web app on the RPi 5. Once hosted on the RPi, I hope to start our testing plans for the game logic accuracy and hint retrieval latency. However, a significant portion of my time next week will be spent on the design report, so these deliverables may extend into Spring Break.

 

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

• This week I made progress on the web app. I set up the Flask backend, began the registration process for Merriam-Webster Dictionary API, cleaned up the NYT Connections Dataset that we will be using, and implemented the custom word upload, hint retrieval, and definition display functions of the web app. For now I used the Free Dictionary API, but will change this to Merriam-Webster API once registration is complete. I also made a Github repo for the web app.
  • Video demo of web app progress so far: link
• Additionally, I made a detailed FSM for the display (web app frontend that interacts with backend and game logic) and a flow diagram for the game flow logic.

• I prototyped a paper cube to get a feel of what size we should make the blocks, and consequentially discovered that 3″x3″x3″ is ideal, and 4″x4″x4″ is the max size holdable in a single hand.

• I also worked on the Design Presentation slides.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

• My progress is mostly on-schedule. I still have to complete work on the frontend, mostly the portion of the web app that has to do with communicating with the embedded controller signals.

What deliverables do you hope to complete in the next week?

• Next week, I hope to have the Merriam-Webster dictionary API up and running, implement the “existing puzzle retrieval” method for the web app, implement the remaining frontend pages, and hopefully try to host the web app on the RPi if possible.

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

• This week I made a Figma board for our web app that shows all the different pages and the different flows that the user may go through.

• I finalized the resources and APIs we will need for the web app after doing some research about different databases and web app backends. Ultimately after taking a look at the functions and load of the web app, I decided on Merriam Webster Dictionary API, Flask for backend, React for frontend, and CSV file or SQLite for database.
• I also started coding up the frontend for our web app in React and Typescript, creating the reusable components that we will be using for the different pages and finishing most of the pages’ front end.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

• My progress is on schedule.

What deliverables do you hope to complete in the next week?

• Next week, I hope to have a cleaned NYT Connections Dataset for our MVP in the form of a CSV file.
• I also want to make more progress on the web app frontend as well as setting up the Flask backend.
• I also hope to have the basic game logic planned out (pseudocode + flowchart).