Posted on

Alexis’s Status Report 4/26

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I gave the final presentation, worked on the poster, helped test out and debug some of the changes made to the webapp and gameflow, added labels to the grid so that users can know positions of the block on the grid. I finished the poster and all that we need to do is update the testing and verification values to what we have been doing this week. Also, I took a look at possibly adding neopixels inside our blocks and making it programmable to the colors itself. Another thing that I am thinking of adding is a 3D printed frame to cover the LCD’s glue to the top portion of the block. With the addition of sound effects I created a plan for how to construct the housing for the speaker to be the block logo that we have for Connexus. I am looking to sew some legs to give the housing for the speaker some character that would add more charm to our project. In terms of hardware I was trying to come up with different ideas for more stronger alignment and will be experimenting with these soon.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is on schedule and with the poster out of the way, we have to come up with a draft for the video.

What deliverables do you hope to complete in the next week?

I will finish user testing, start outlining and then finishing the video, starting and finishing the final report, and finishing up all the additional cosmetic touches.

Posted on

Wen Hui’s Status Report for 4/26

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • I worked on helping Nicole with the integration of the new webapp with the embedded controller, especially additional features like the 8 or 16 block options. There were also some modifications to the embedded controller code structure required, such as returning the grid positions that were missing / not ACKed so that the webapp can display to the user which block to adjust in order to ensure correct uploading.
  • Another feature I worked on this week was the autoscrolling of the row category LCD. Some of the category names are really long (eg.
    SINGULAR OF THINGS SEEN IN PAIRS) and exceed the width of the LCD which has 32 characters. I had to implement multiple screens of text on the LCDs so that it is able to cycle through all the words and display the full category name without overlapping. Since I have to run the autoscrolling for all the 4 LCDs and have the program running together with the actual main program, I implemented this using multithreading. I spawned a thread for each of the LCDs that were in charge of making sure the text on the LCDs were scrolling. The main thread will then update the message variable for each LCD depending on whether display_category is being called by the webapp. With all these changes, I made a class called lcd_controller that encompasses all these functions. This will allow the seamless integration of the embedded controller with the lcd on each row.
  • I also conducted testing on the font size for the LCD on the Picos. I tried loading words of lengths 1 to 25 (the maximum length of words in the database) and ensured that the word(s) showed without being truncated and were a reasonable font size to the user.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is on schedule. I plan to work on some final user testing next week, before focusing on deliverables like the video and report.

 

What deliverables do you hope to complete in the next week?

In the next week, I hope to work on getting a few more user testing done. I also plan to work on writing out the report in detail given the testing we have done. I will also work with my teammates to improve the aesthetic of our project and reflashing the Picos with a different display color that is more distinct to the user.

 

Posted on

Nicole’s Status Report for 4/26

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • This week I enhanced the web app with several new features. I integrated and debugged all these features so that they work with the hardware on the RPi.
    • I enhanced the “missing block” feature to report the position of the blocks that are missing or have a bad connection.
    • I added sound effects and color changes for the submission checking pages to draw the user’s attention to the web app screen. In user testing, we noticed that users would miss the answer-checking result screen because they would be too focused on the blocks and not looking at the web app.
    • I implemented and integrated an “easy” mode which is 8 blocks and 2 categories, as opposed to the regular 16 blocks and 4 categories.
    • I implemented and integrated a feature that lets us play a puzzle given a specific game ID, which we can use to test different puzzles during user testing.
    • I added a “super hints” feature where you can see 1 word in each category (chosen randomly).
    • I fixed some frontend bugs where the mistake count would disappear in certain cases.
    • I added a “game status” check on each page that checks the number of categories found and mistakes remaining and transitions to the lose/win screen if needed. This should minimize the chance that we somehow get stuck in a state where all categories have been found but the web app doesn’t transition to the win page, or the situation where the mistake count goes negative and still allows the user to play.
    • I created a database (giant json file) of pre-retrieved hints for each word (~5k) generated from Gemini. This way we can cut down on the hint retrieval time since we can just fetch them from the database instead of generating them in real time.
  • I started researching using neopixel strips with the on-blocks picos to enhance the aesthetics of the game.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

  • My progress is on schedule. I’ve pretty much completed all my tasks and will just be focused on helping my teammates with enhancing the aesthetics/cosmetics of the project. 

What deliverables do you hope to complete in the next week?

  • Next week I will complete the video and final report. I also hope to conduct 5 more user tests with our newly enhanced web app features. If time permits, I hope to get neopixels working on the blocks as well.

 

Posted on

Team Status Report for 4/26

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

  • The most significant risk that could jeopardize the success of the project is the reliability and quality/durability of the pogo pins. We discovered that some of the individual pegs of the male pogo pins become permanently partially depressed (i.e. the spring mechanism degrades and doesn’t push the pogo pin up to full height) perhaps during soldering. This results in a bad connection between the grid and the block no matter how hard we try to align the 2 pogo pins, thereby prohibiting us from playing the game at all since we absolutely need the pogo pins to get that wired connection for our UART communication. To manage this risk, we have a few extra male pogo pins on standby (pre-wired) in case something goes wrong or one of the grid pogo pins degrades. However, we have found that pushing the male pogo pins up from the grid more and pushing the block’s female pogo pin out more can help attain a good enough connection for these defective pins, so our contingency plan if we cannot replace a pogo pin is to rework the grid/block construction slightly to allow these pogo pins to stick out a bit more. For now, it seems like the connections are working pretty well and are relatively stable after we swapped out the bad pogo pins.

 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

  • Yes, we had to supplement the Merriam-Webster hint retrieval with generated output from Google’s Gemini LLM since Merriam-Webster failed to retrieve definitions for a small percentage of the words in the dataset. This was necessary because there were some games wherein more than 1 word had no hints retrieved at all, which is extremely detrimental to our primary use case of making the game a better experience for novices/ELLs who rely on the hints to gain more background knowledge about specific English words. The cost of this change was in terms of latency. Using Gemini boosted our hint retrieval rate to 100%, and the LLM hints were generally more readable/distinct than the Merriam-Webster ones (i.e. improved quality). However, it is about 15x slower compared to Merriam-Webster. To mitigate this latency cost, we only retrieve hints from Gemini when no definitions or context are fetched from Merriam-Webster. Later on, we decided to pre-generate all the Gemini hints and store them in a local database, so that during program execution we could just read this database instead of making an API call. We then experimented with extending this technique to the whole NYT dataset, pre-generating Gemini hints for all words and storing them in a giant JSON file that we can just read from during program execution. This makes the hint retrieval O(1) and has the added benefit of readability from the LLM output.
  • We decided to add sound effects and colors to the web app to draw the user’s attention to it during the answer-checking stage. This was necessary because during user testing, some users would be too focused on the physical blocks during answer checking and wouldn’t look at the screen to see if they were correct, incorrect, or one away. The cost of this change was not much in terms of latency; we just had to write more code.
  • We decided to add a “loading” page for the answer checking stage since the latency from button push to transitioning to the appropriate result screen was a bit long, so the user couldn’t tell if their button push was actually registered or not and would sometimes press the button again, resulting in a erroneous double submission. The cost of this change was not much in terms of latency; we just had to write more code.
  • We enhanced the “missing block” feature to display on the web app the specific positions of the blocks that are missing or have a bad connection, so that the user can manually fix these before resubmission. This was necessary because the alignment of the pogo pins between the block and grid was quite difficult for users to get on the first try since we had not implemented magnets on the blocks/grid, so if they were misaligned we would have to look ourselves on the backend to see the indices of the bad blocks. The reason we did not implement magnets is because each grid and each block’s pogo pins are slightly different in terms of positioning, so we were having a difficult time coming up with a way to get the magnet placement to be interchangeable between all the blocks and grid positions. The cost of this feature was having to rework the code a bit to relay this information, and in terms of latency we had to query all the blocks in the grid/row before returning to the web app instead of just stopping at the first failure, so the latency was slightly increased. To mitigate this, we decreased the delay from 0.5 to around 0.1-0.3 seconds for the UART querying, which made our UART communications much faster. 
  • We want to have our row LCDs be able to “scroll” so in case a category name is too long to fit on the whole row LCD, it doesn’t get cut off and the user can still see it. This is needed since the row LCD is the only way the user can see the category name if they guessed correctly.
  • We added a “super hints” feature to help users who were really stuck, even with the provided hints. This super hints feature gives 1 word in each category. We thought this would be necessary to help make Connexus a less frustrating experience for players compared to the original game. 
  • We added an “easy mode” feature that is only 8 words and 2 categories. This mode is useful to novice players since it is much easier to win the game this way, and helps the user to get in the right mindset that is needed for the normal game without all the frustration and difficulty of the full 16 word mode.

Provide an updated schedule if changes have occurred.

  • All the changes except have been implemented, tested, debugged, and integrated into our product. Thus, the updated schedule just includes us working on potential aesthetic enhancements like 3D printed frames for the block LCDs, a custom housing for our speaker, and neopixels for the blocks.

This is also the place to put some photos of your progress or to brag about a component you got working.

  • Please see our photos and videos here.

List all unit tests and overall system test carried out for experimentation of the system. List any findings and design changes made from your analysis of test results and other data obtained from the experimentation.

  • Hint retrieval quality test
    • Findings:
      • Merriam-Webster does not retrieve any definitions/contexts for a word 1.57% of the time
      • There are 5.37% of games w/ >1 missing word (“bad” game)
    • Design Changes:
      • Supplement hint retrieval with LLM API calls
      • Supplementing w/ Gemini, we are able to retrieve hints 100% of the time
  • Hint retrieval latency test
    • Findings:
      • Merriam-Webster has a latency of 0.11s per word
      • Gemini Flash 2.0 has a latency of 1.5s per word
      • [Merriam-Webster] Average “good” game latency = 1.81 sec
      • [Merriam-Webster + Gemini] Average “bad” game latency = 4.28 sec
      • Average game latency = (1-(33/614))*1.81+(33/614)*4.28 = 1.94 sec
      • We will meet our 3.2 sec latency requirement ~95% of the time
    • Design Changes:
      • We pre-generate all the Gemini hints and store them in our own local database so that we don’t have to make API calls live when the program is running, we can just fetch the hint from our dataset.
  • Game logic accuracy test
    • Findings: Answer checking logic was 100% accurate
  • User testing
    • Findings:
      • More often than not, some blocks would be disconnected or misaligned, resulting in missing ACKs during UART communication. The user would not be able to see which blocks were being problematic.
      • Users wouldn’t realize the web app is showing the result of their submission and would miss it.
      • The puzzle was sometimes too hard even with hints.
    • Design Changes:
      • Displaying the positions of the missing blocks on the web app.
      • Colors and sounds during answer checking.
      • “Super hints” for each puzzle which gives 1 word in each category.
  • Battery Life of Blocks
    • Findings: Each battery lasts around 6 hours of continuous use
  • Weight and Size of Blocks
    • Findings:
      • Each block weighs around 181g and is 3.3” x 3.3” x 3.3”
  • UART Tradeoffs during upload_words
    • Delay (latency) vs Accuracy
      • Findings: Ideal combination was at 0.2s for total upload latency of 3.2s but 100% accuracy
    • Baud Rate vs Accuracy
      • Findings: 
        • Baud rate > 115,200 are able to received within one try (0.2s delay)
        • Baud rate < 115,200 received within 2 tries (0.4s delay)
  • UART Send & Receive
    • Findings: 
      • 100% accuracy
      • 200ms latency on average for each word
  • Answer Checking E2E
    • Findings:
      • Average incorrect latency: 0.9419006s
      • Average correct latency: 3.0390748
      • Average latency: 1.9904877s
      • Average accuracy: 97.50%
        • Sometimes accidental double submissions or failed transitions that cause inaccuracies.
        • Sometimes the category display on a row LCD failed.
    • Design changes:
      • Added a “get status” function to every web app page that involves gameplay so that we are guaranteed to transition to win/lose if game end conditions are satisfied no matter what state.
  • Word Upload
    • Findings
      • Average latency: 3.231673384s
      • Average accuracy: 97.50%
    • Design changes:
      • Added the position reporting of the missing block to the frontend.
Posted on

Team Status Report for 4/19

 

What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

Pogo pin defects

  • Some of the male pogo pins have slight manufacturing defects of or from soldering wires to the back of it. Some of the pins on the pogo pins are uneven and different height, hence even if the block is level with the grid, the pogo pins will not be well connected between the female pogo pins (on the block) and the male pogo pins (on the grid). This could be the reason why some of the blocks did not ACK back to the RPi during transmission of commands.
  • In terms of contingency plan, we have resoldered and swapped out the ones that are uneven on the grid with working ones. We also made sure to test every block with every position on the grid to make sure all possible combinations are working for pogo pins connections.

Sturdiness of the grid

  • The grid broke along the joints between two pieces during our testing process. We plan to reinforce it with wood glue again and potentially glue another piece of wood at the connection point between the two pieces of wood to ensure that the connection is secure.
  • Circuitry was also a possible point of failure. Since we have to use jumper wires to connect the LCD, buttons and male pogo pins to the RPi, the connectivity might not be the best and we encountered instances of the wires coming loose from the connection, causing malfunctioning hardware components. We have decided to solder the connection between two wires and tape them with electrical tape to reinforce it.

 

Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward? Provide an updated schedule if changes have occurred.

The height of the grid was adjusted and there is more space for the Rpi and circuitry now. This is because the circuitry turned out to be more wire heavy than expected which doesn’t fit entirely under the lid of the grid. This is more of an aesthetic change so no major schedule changes is expected. We still plan to collect all the user testing data this week and have the lid ready by the final demo.

We also changed from a Flask app to a FastAPI app for the backend of the web app. This is because after constructing the final grid and seeing that there were still alignment issues with the pogo pins that caused issues with communications not being received, we decided we wanted to notify the user of specific problematic grid indices so that they can manually adjust the alignment of specific blocks. In order to expose this info to the user, the backend needs to initiate one-way communication with the frontend, which requires the use of something like web sockets. However, the asynchronous web socket frameworks available to Flask are incompatible with the RPi’s GPIO library due to monkey patching, so we changed the backend app to a FastAPI app instead, which supports python’s native asyncio web socket package through uvicorn, which is compatible RPi.GPIO. The cost of this change was time and effort in terms of code translating/rewriting, but we were able to get it done this week so there is no change in schedule resulting form this design implementation change.

This is also the place to put some photos of your progress or to brag about a component you got working

  • Completed the production of grid and 16 blocks, together with soldered perf boards which have been tested to be working, combined with the new webapp. We also attached the pogo pins onto each 3d printed holder on the grid and glued them on. We also wired the buttons, LCD and pogo pins to the perf board and tested their functionalities individually. We have the ability to play the whole game without issues now! More photos can be found here.
  • Webapp has the added 1 away feature which tells users if their guess is one away from the correct guess.
  • Completed latency testing and webapp statistical analysis for hint retrieval on the entire dataset.
  • In the process of doing user study on ELLs and novice players this weekend.

 

 

Posted on

Alexis’s Status Report 4/19

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

This week I worked on laser cutting the grid, making 3D printed holders for the pogo pins on the grid, assembling, integration of our final product, verification testing and user testing. For the grid I was able to cut all the pieces out and wood glue it together, letting it set for 24 hours before we started playing around with it. Before that I had used scrap cardboard to ensure that everything would fit together and that it was an appropriate height. I found that I didn’t add enough tolerance and therefore had to modify some dimensions. I also initially had engravings to make way for the wires but it did not cut deep enough and took too long to do the engravings so we ended up laying out the wires in the back nicely and taping it down. I also printed out the holders for the pogo pins which helped to align it to its position on the grid ensuring that each pin was straight and flush to the board. These proved to be super helpful when installing the pogo pins and minimized variability in its positioning. I also added standoffs as I realized there would be some wiring on the back side that I didn’t want to get squished when laying flat on the table. I then worked on integrating with the soldered perf board and solidifying all our connections: removing jumpers when possible, soldering the joints and heat shrinking it to provide insulation. There was quite a lot of debugging and retesting connections as it got jumbled while migrating the test board to the grid, however we were successfully able to finish and start testing. I also discovered that there may be manufacturing defects with some of our pogo pins as there are times when the male pogo pins don’t come to the same height and one is sunked down (look here for reference) which may also attribute to some loose connections and why we may have troubles receiving ACKs sometimes. As for verification and testing I worked on uploading 16 different words to each block and at different positions. The way I did this was first starting out with block 1 on position 0, block 2 on position 1 and so on. I then wrote some code to recording the start and end of uploading a word associated with a particular position. I noted all the times and shifted all the blocks down by one so that now I had block 16 on position 0, block 1 on position 1 and so on. I continued this for 16 times and was able to see satisfying results with around 200 ms on average to upload varying lengths of words. I also conducted user tests where we set up the user to play NYT Connections and then Connexus and take a survey. I recruited about 5 people for user testing. I also made sure to weigh and check the size of the block which we found that the weight was 181g and so far everything is meeting our original design requirements. I also am working on graphs for displaying testing metrics for the final presentation.

Look here for more photos and videos!

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is now on schedule as I have finished the grid and integrating everything. The only thing that is left is more user testing.

What deliverables do you hope to complete in the next week?

More user testing and possibly making minor cosmetic changes. I will also be starting on the poster and looking at comments for the final report.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

The new knowledge I have acquired is all the mechanical knowledge such as using the Epilog laser cutters and using Solid works to get the correct dxf files. I also learned that a lot of my designs were a bit of trial and error, I tried to use scrap material where I could to practice and then moved onto our actual product, but you can see that hear and there some of the mess-ups I had along the way, but once I got the hang of it everything became really easy. I also learned more about micropython and creating a protocol for the picos to communicate with the RPi. I enjoyed making a class abstraction to better structure my code so that it was easy for integration. I also became a lot more comfortable at soldering as repeating the same actions while scaling became a routine for me. I also learned how challenging it can be to scale as now there are so many moving parts and variability in places you didn’t know existed, but I think that my team and I came up with a good plan and we have been executing it well to ensure organization such as labeling the blocks and modularizing testing to ensure connectivity and functionality.

Posted on

Wen Hui’s Status Report for 4/19

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • I also re-flashed all the Picos with the new code which includes error handling in cases where the received word does not include valid alphanumeric characters. It turns out that when an invalid string to be parsed is sent to the Pico, an exception is thrown and the Pico is unable to continue even if the right data is sent to this. Hence this requires us to turn on/off the Pico. With this modification, all the Picos can function with multiple repeat games without having to be turned off.
  • I verified the functionality of all the connections of each pogo pin on the grid and had to replace some of the jumper cables on the grid that were not producing electrical connections. One of the Picos also had a malfunctioning UART pin so we had to resolder that as well. This helps to ensure that any errors that could arise is due to the perf board itself instead of the grid.
  • I completed the soldering of the perf board PCB and integrated it with the 16 block test bed on Monday. There were a lot of issues encountered when soldering the perf board especially with weak connection for some of the pins, causing bad data to be transmitted or the blocks to not ACK all the time. I also encountered a strange error where block positions 0 and 2, and 1 and 3 will get uploaded at the same time. After repeated debugging, to figure out if it was the blocks, pogo pins or the perf board circuitry that was the issue, I found out that it was due to a malfunctioning GPIO pin in the RPi that was controlling the mux, causing there to be some lag when selecting the UART lines. After switching out the MUX select pins, I was able to implement successful uploading of words to all 16 blocks. (Refer to test video)
  • After Alexis laser cut the plywood for the grid, I helped with testing the electrical connections and debugging the circuitry to ensure that it all still worked with the actual grid instead of the test bed.
  • I also worked on the latency testing but modifying the embedded controller code to be able to specify the duration of latency delay to allow us to vary it for testing purposes.

 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

My progress is on schedule. I plan to work on getting all the quantitative results for latency testing over the weekend in time for the final presentation.

 

What deliverables do you hope to complete in the next week?

In the next week, I hope to implement some extensions and aesthetic improvements to the project. I wish to ensure that the colors of the LCD on the Picos are visible. I also hope to try out multithreading to allow multiple words to be uploaded to different blocks concurrently, speeding up the process of the uploading phase.

 

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge? We recognize that there are quite a few different methods (i.e. learning strategies) for gaining new knowledge — one doesn’t always need to take a class, or read a textbook to learn something new. Informal methods, such as watching an online video or reading a forum post are quite appropriate learning strategies for the acquisition of new knowledge.

I was working on the embedded controller portion of the code, including the Rpi and Pico. I had to find out how to control all the components via Python such as the Sparkfun LCD, detecting button press, UART communication with each block and the RGB Waveshare LCD.

I referred to online documentation about Sparkfun LCD for the LCD portion of the code. I also looked at Medium articles and watched online videos to figure out how to enable UART1 on the RPi. For the serial communication between the RPi and Pico, I looked into online forum posts for references on how to structure my code, potential suggestions for waiting for the serial transmissions and delays necessary to ensure that sufficient time was given for UART transmission. I also looked into online forum for error handling processes. I also searched up Arduino forum to figure out the button circuit to use for our submit buttons. For the button callback function, I looked into the Rpi HQ guide online for how to integrate the button into the RPi program without blocking the rest of the program flow.

I also had to debug the perf board after soldering it, hence I looked into the datasheet for the MUX to figure out which pins have to be connected to ensure the MUX is working well. I also looked into online forums to figure out possible ways that could prevent a block from ACK-ing, deriving the solutions of: poor connection between ground of Pico and UART and loose wires of TX/RX.

 

Posted on

Nicole’s Status Report for 4/19

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • This week I completed our hint retrieval latency and accuracy test. I tested our entire database of puzzles, gathering timing and performance accuracy metrics with regards to the hint retrieval. I recorded how many words from each puzzle did not have any definitions nor context retrieved by the merriam webster dictionary API, and I timed how long each API call took. I then analyzed this data to gather statistical performance metrics in terms of average latency of hint retrieval and the quality of the retrieval. 
  • I found that there were puzzles that have unretrieved hints for multiple words, which is unideal for the use case of our project, so supplemented our dictionary API hint retrieval with hints generated from an LLM (in our case, gemini flash-2.0 model). I then tested the entire database again with this new capability and did a quality/latency trade-off study on the different LLM models available and different mixtures of APIs.
  • I helped to construct the final grid. I helped come up with a solution to our pogo pin alignment problem and also helped in physically soldering connections on the circuit boards and other wired components on the grid. I then helped to debug the final circuits as there were some accidental shorts. 
  • I rewrote the web app to use SocketIO in place of the POST requests I was mainly using in our previous implementation. Previously, I was having trouble with getting the sockets to work consistently on the RPi as it seemed some signals were just never received. I had implemented a work-around using the return status codes of POST requests instead of socket signals, but after testing out our final grid implementation and seeing that we still suffered from alignment/connection issues, we decided that we would need to implement an error message feature that would require the use of sockets for one-way server to client communication. I was able to get a flask-socketio implementation to work on my mac laptop using a greenlet (eventlet) framework (I figured out that the issue I was facing before was due to the synchronous nature of the socketio function calls, but when it came time to integrate it onto the RPi, I ran into issues.
  • It turns out that eventlet/gevent (greenlet frameworks) are incompatible with the RPi.GPIO library since they require monkey-patching in order to enable asynchronous socket capabilities. I needed monkey-patching to get the sockets to work, but it turns out this implementation is not possible to run on the RPi. Instead of eventlet, I researched other alternatives and ultimately found that uvicorn, an ASGI web server, along with python’s native asyncio package should work with RPi GPIO. However, this package does not have Flask support, so I rewrote the entire Flask backend as a FastAPI backend instead. I then translated the appropriate frontend socket logic as needed, then reintegrated with the RPi and found that it worked great.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

  • My progress is a little behind schedule because we were having lots of issues during integration for the final product. As a result, I didn’t get to do as much user testing this week as I wanted to. However, now everything (minus aesthetics) is basically done and ready for user testing, so I will block out time next week to finish the user testing that I didn’t get to do this week.

What deliverables do you hope to complete in the next week?

  • Next week I hope to complete our user testing. I also want to get started on the poster and final report. Additionally, I will work on making our project more aesthetic (think about painting/branding).
  • I also want to implement a feature on the web app where it displays the position of the block that is missing instead of just generically saying that a block is missing.

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

We recognize that there are quite a few different methods (i.e. learning strategies) for gaining new knowledge — one doesn’t always need to take a class, or read a textbook to learn something new. Informal methods, such as watching an online video or reading a forum post are quite appropriate learning strategies for the acquisition of new knowledge.

  • I had to learn about backend frameworks such as Flask and FastAPI, CORS policies, as well as using web sockets in python. To learn about these frameworks, I watched many tutorial videos on building React + Flask/FastAPI applications, and read deep into the documentation about the SocketIO/Flask-SocketIO and asyncio packages for python. Additionally, I had to read many stack exchange and discussion forum posts to diagnose specific issues I was having with integrating the web app on the RPi and with CORS issues.
  • I also had to learn how to use the merriam webster dictionary API and particularly how to parse the json response from it. In order to do this, I read the github pages of other authors who created wrappers for the merriam webster API to get inspiration.
  • Finally, I had to learn about using generative LLM APIs such as gemini. To do this, I read deep into the documentation and read through their provided example programs.
Posted on

Nicole’s Status Report for 4/12

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • This week I fabricated and tested all the internal circuitry for the new 8 blocks.
    • I soldered all battery holders to adapters, soldered headers onto all picos and muxes, cut charging ports into battery holders, and tested all picos, waveshare LCDs, batteries, battery holders, and adapters for functionality.
  • I changed the web app to handle 4 rows by using the full NYT Connections dataset and using their startup word placements instead of a random shuffle. I also implemented the “one word away” feature.
  • I helped to fabricate the test bed for 16 blocks. I laser cut cardboard platforms for the male pogo pins, cut all the wires needed, and helped to tape everything onto a text board.
  • Additionally, I soldered wires for the buttons and row LCD displays, and did a little soldering to help with the transfer of using a breadboard to using a soldered perf board for the grid circuitry.
  • I went through the NYT connections database and picked 2 puzzles of similar difficulty that contained words involving knowledge of American cultural references and double meanings to use for our user testing.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

  • My progress is a little behind schedule because we were having some issues with the 16-block grid working consistently, so I did not get as much testing as I would have liked. To remedy this, I will write scripts to perform my latency testing so that once the grid is ready, I can just run my scripts and get the testing done as fast as possible.

What deliverables do you hope to complete in the next week?

  • Next week I hope to complete my verification tests and gather user testing data.
  • If I have time, I will also try to improve the text cleaning for the dictionary api and potentially look into using gemini api for words or phrases that don’t show up in the dictionary api. 

Now that you have some portions of your project built, and entering into the verification and validation phase of your project, provide a comprehensive update on what tests you have run or are planning to run. In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements? Verification is usually related to your own subsystem and is likely to be discussed in your individual reports. Validation is usually related to your overall project and is likely to be discussed in your team reports.

Tests that I have run:

  • Game logic accuracy
    • Input: Sets of submitted words
    • Output: 100% correct answer checking
    • I played 10 games. For 5 games, I tested 2 combinations of correct submissions and 4 combinations of wrong submissions. I made sure that 4 wrong submissions resulted in a game loss. For the next 5 games, I tested 2 combinations of wrong submissions and 4 combinations of correct submissions. I made sure that the 4 correct submissions resulted in a game loss. For all games, I made sure that the correct submissions were registered as correct and the wrong submissions were registered as incorrect. I also verified that the mistake count was decremented and reset properly, and that submitting the same category multiple times would not result in a premature game win. For each of the 10 games, I made sure that at least 1 wrong submission was a “one away” submission, and checked that the “one away” message was displayed properly.
    • From these tests, I was able to verify that the game logic was 100% accurate, which is imperative as incorrect game logic will result in the user not being able to finish the game properly since each successive submission depends on what words have been “eliminated” by previous submissions.

Tests I am planning to run:

  • Hints Retrieval Latency
    • Input: Load 20 puzzles
    • Output:3.2s latency/puzzle
    • I plan to write a script that will run on the RPi to test latency of hint retrieval. The script will load 20 puzzles. For each puzzle, I will time how long it takes to retrieve/generate the puzzle, retrieve hints via calls to the dictionary API, and parse and store these responses into the game controller data structures. This will give us an idea of how long it takes to gather hints for all of the words in a puzzle, and make sure that it is not too long so that we can meet our latency goals and maintain user attention.
  • Hints Retrieval Quality
    • Input: all words in the NYT Connections Archive dataset
    • Output: 
      • 95% retrieval rate of definitions
      • no more than 1 word’s hints unretrieved for each puzzle
    • Aside from the latency, I also want to test the quality of the hint retrieval itself. I will write a script to loop through every word in the dataset and record the number of times where no definitions or context/examples were found for a word. I want to make sure that definitions/examples are retrieved at least 95% of the time. Additionally, for each puzzle, I want to make sure that no more than 1 word in that puzzle does not have hints available. If I do not meet this goal, I will consider options like generative LLM APIs such as Gemini to provide hints for missing words. It is important that the virtual companion is able to retrieve hints reliably since otherwise it will be completely useless to the user as a feature.
Posted on

Alexis’s Status Report for 4/12

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

I have been working on assembly for the 8 additional blocks that we had, mostly laser cutting, hot gluing, spraying the blocks, and soldering all the pogo pins with the 6 pin header since these pogo pins were surface mounts. I also helped Nicole to create a 2nd test bed and created a “bridge wire” from the main board to the mini-breadboard of the second testbed so that we can daisy chain and get the wires we needed from the RPi5. I also worked on planning out how we would cut out all of our wood to assemble the grid with a CAD and researched into how another team was incorporating their pogo pins into their design. I came up with a new 3D printed pogo pin holder to ensure that the pogo pins are placed onto the grid in an organized and systematic way so that we can easily assemble the grid without variability. I did box joints for the grid so that we can apply wood glue to hold it in place and I also made the cut outs for the button and the LCDs. I made some design changes which was to reduce the spacing between rows and move the row LCDs to the right side of the board above the buttons. I also helped come up with some trade-off studies to prove our design choices as well as created a survey that we will ask users to fill out during user testing.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

I am currently slightly behind schedule as I was supposed to finish the grid but we expanded to 16 blocks and I also had to come up with a solution for the pogo pins. Since I have the files now I will be doing an initial cardboard cut out to confirm the design and then I’ll be able to assemble the grid itself.

What deliverables do you hope to complete in the next week?

I will have the grid assembled, help debug the 16 block set up, and complete the user testing / verification tests that I have for the parts that I am responsible.

Verification Testing

I will be performing the battery life endurance test which we have already been doing informally and the blocks’ batteries have been able to last for weeks with several days of 2 – 4 hours of being on while we are testing. I will also be working on the trade-off tests for checking different delay times for the upload_words function as well as experimenting with different UART baud rates to show how we came to the decision to use certain specs. The blocks were my subsystems that I am responsible for and I will be weighing and measuring each block to ensure that it meets the requirements of being <4"x4"x4" and <400g. We purchased blocks that were already 3.3"x3.3"x3.3" so the measurements are already satisfied and I will just need a scale for measure but I also did this informally and it seems to meet this requirement. I will also be working on UART Send & Receive to test how 10 words are sent from RPi5 and using the time module we will collect data on the latency it takes from the upload word query to receiving an ACK on RPi 5 side. This will be performed on all of the blocks. In addition I will my other teammates by writing any scripts that help us collect our data and I will be recruiting people to take part in our user study as well as come up with more questions that would allow us to show how Connexus compares to NYT Connections.