Team Status Report for 3/8

A was written by Wen Hui, B was written by Alexis and C was written by Nicole.

Part A: … with consideration of global factors. Global factors are world-wide contexts and factors, rather than only local ones. They do not necessarily represent geographic concerns. Global factors do not need to concern every single person in the entire world. Rather, these factors affect people outside of Pittsburgh, or those who are not in an academic environment, or those who are not technologically savvy, etc.

Connexus has a virtual companion that provides hints to the user during gameplay. This simplifies the process of searching through the internet for American culture references and word definitions themselves. It helps to bridge the gap in this digital game of Connections for users who might not be technologically savvy or well versed with the internet enough to search for hints on their own. Our game is built such that the user can just interact with the physical game without having to open their own digital devices at all, making it inclusive to individuals of different technological skill levels. Additionally, Connexus helps to make Connections more accessible to everyone, even for people who might have low attention spans when interacting with digital devices or those who are more tactile learners. Our game makes Connections a game that stimulates the player’s sense of touch instead of just sense of sight, allowing them to be more engaged and focused in the game.

 

Part B: … with consideration of cultural factors. Cultural factors are encompass the set of beliefs, moral values, traditions, language, and laws (or rules of behavior) held in common by a nation, a community, or other defined group of people.

Connexus was created to address the frustrations of new and non-native players by offering an interactive, educational, and culturally adaptable gameplay experience that NYT Connections currently is lacking. This game is designed for more inclusivity across cultures by bridging language gaps while still maintaining the challenge and fun of the original game especially for English Language Learners (ELLs). It addresses the need for accessibility and engagement by helping players who would struggle with ambiguities in word meanings due to multiple word definitions, cultural references or elitist knowledge assumptions as the creator of the game mentions. This is accomplished with the virtual companion that will retrieve hints from an API and also it supports customizable puzzles that allows for players to come up with their own puzzles either for educational use or for fun. By allowing for a personal twist, you can create different cultural contexts as well which would make the game more inclusive. Also, Connexus provides physical and movable blocks that would shift from a purely digital interface to a more engaging gameplay that would be beneficial for different learning styles.

 

Part C: … with consideration of environmental factors. Environmental factors are concerned with the environment as it relates to living organisms and natural resources.

Connexus doesn’t have too many significant environmental considerations other than the power consumption and physical materials. Each block has an on-board microcontroller and LCD display that needs to be powered by a battery. In order to reduce the amount of waste that our project generates in the form of dead batteries (which could be harmful to the environment if not disposed of correctly), we opted to use rechargeable batteries with an on/off switch. This way, the same batteries could be used over and over again until they run out of recharge cycles, and energy can be conserved when not in use by switching the batteries off. 

Additionally, the RPi that is hosting the web app will be consuming power as it makes API calls to the dictionary API and coordinates the hardware and software components of our system. These processes consume power, so we want to minimize the communication that the web app has to make with external APIs and the hardware subsystems (i.e. blocks). Writing efficient code based on interrupts and requests as opposed to constantly polling for responses is one way we plan to minimize communication. Furthermore, we “prefetch” all the dictionary API calls/responses at the start of the game and then access these locally in the program, as opposed to making repeated API calls every time a user navigates to a word’s “hint” page, so as to minimize our interaction with the external API. All these measures are in an effort to not waste unnecessary power.

 

Weekly Questions

• What are the most significant risks that could jeopardize the success of the project? How are these risks being managed? What contingency plans are ready?

The primary risk we anticipate would be the multi block integration with the grid. We have only tested our code logic with one block’s circuit since the rest of our components had not yet arrived. If there are issues with using the multiplexer to interface with multiple UART lines, we will have to explore alternatives. The risk can be managed by modifying the program to utilize other communication protocols like I2C instead. If the latency is too high, the complexity of the UART communication algorithm can be simplified.

Another risk is the secure placement of parts within the block. In each block, there will be the Pico, LCD, battery components and pogo pins. It is important to secure the components to maintain the electrical connection. Initially, we soldered wires directly from the pogo pin wires to the LCD breakout board, but the wires would break easily. Our solution was to solder female pins onto the other side of the Pico breakout board to allow us to plug in wires for the pogo pins. We aim to test the completed blocks with this layout to see if the connection remains secure. Otherwise, our mitigation would be to make 3D printed holders for the components within the blocks to hold them in place.

 

• Were any changes made to the existing design of the system (requirements, block diagram, system spec, etc)? Why was this change necessary, what costs does the change incur, and how will these costs be mitigated going forward?

For the design, we were initially going to use wood to make the grid. However, we looked into procuring wood but realized that it was too expensive for our budget. We decided to switch to using cardboard instead since it was easier to build and cheaper to procure, while still maintaining the rigidity we need for the grid. The cost for our project will be reduced with this change.

 

• Provide an updated schedule if changes have occurred.

For our schedule, we are on track and there has not been major changes. We are not going to build the actual grid yet, but instead focus on prototyping a platform to place the grid pogo pins on for testing. The plan now is to focus on having fully built blocks to test the end to end functionality of multiple blocks with the grid circuit.

 

• This is also the place to put some photos of your progress or to brag about a component you got working

In the past week, we have fully built one block including laser cutting all the cutouts for the pogo pins and charging port, and placing all the soldered components into the block. We have also soldered all the components to prepare them for the remaining blocks, only being left with the laser cutting of the blocks before they can be fully tested.

On the software side, we have fleshed out the design for the Embedded Controller, Game Controller and Web App functions and how they interface with one another. We have also written out all the code functions we need for the RPi and Pico and tested a single Pico with the RPi.

We also have a functioning webapp with Merriam Webster API integration and was able to simulate submission and loading of words from the webapp side. There is also exposed API endpoints for the embedded controller to trigger when the submit button is pressed.

Everything is ready for a primary end-to-end integration and testing next week!

Wen Hui’s Status Report for 3/8

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • I finished the designing and coding of the embedded controller and UART controller (helper class within embedded controller to handle UART communication by abstracting away the processing of retrying + delays) on the RPi. I wrote the functions send_query, send_word and send_color within the UART controller that handles sending of the respective commands to the Pico. I wrote the functions on the embedded controller which interfaces with the game controller (the game controller will be written by Nicole along with the web application). I thought about and wrote the embedded controller submission logic by integrating GPIO with a callback function that sends a POST request to an endpoint that is specified by the web application. I tested the multiplexers which were newly acquired and integrated it to embedded controller logic for selecting the grid position to read/write from. 

  • All the code for the embedded controller to handle game play and user interaction is complete. I also finished testing all the functions within the Rpi within the limited capacity of a block and a grid circuit built on a breadboard.
  • I also helped with the replication of the other 7 blocks by soldering header pins onto components such as pogo pins and Picos.
  • I started looking into the design for the grid system and asked Techspark about wood materials for laser cutting. We found that purchasing wood is too expensive so we are switching to using cardboard instead.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

  • I am currently on schedule with an upcoming task of testing 2 blocks once they are fully built with an independent power source within the block.
  • We have decided that I will focus on prototyping the code for a single block first before working on the grid design. I intend to test the grid circuit on the breadboard for now while I ensure that all the blocks we make are functioning.

What deliverables do you hope to complete in the next week?

  • I hope to start testing the integration of the embedded controller with the game controller (especially for submit button). I will test if the POST request is being sent from the embedded controller to the game controller.
  • I also hope to test out a fully built block and whether it interacts well with the grid breadboard circuit.

Wen Hui’s Status Report for 2/22

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).

  • I built the circuit for the RPi on the breadboard for testing purposes. This included the pogo pins, LCD (for row category display) and button (for answer submission). 
  • I also tested the components we received including the pogo pins, regular LCD for row category display, RGB LCD for block word display and the button individually to make sure they were working well as expected. For the button, I utilized the RPi built-in GPIO functionalities to detect the button press by using a callback function. For the regular row category LCD, I utilized the Sparkfun QWIIC SerLCD library to interface with the LCD, allowing me to change the backlight color (when we display the answer) and the text on the screen. The RGB LCD for block word display was purchased from Waveshare and there were starter codes found on the Waveshare website which I referenced to set up the interface between the LCD and the Pico.
  • I also worked on implementing a basic UART communication between one RPi and one Pico via the pogo pins. To program the Pico, I installed Thonny on my computer in order to program the Pico using MicroPython. On the Pico, I utilized the machine library to read from the UART pins of the Pico. On the RPi, I utilized the pyserial library to read from the UART pins of the RPi. 
  • The main issues I was concerned about were transmission latency and accuracy. It seems like latency wasn’t an issue because one cycle of transmission (including sending and waiting for ACK) includes a 100ms delay. This will ensure that in the worst case scenario a block is not present, the RPi will detect this within 300ms. In the future, I might adjust the delay time period to allow for faster communication between the RPi and the Pico. In terms of transmission accuracy, I encountered issues where the Pico would not receive the entire word from the RPi or it might mistake a word that was transmitted twice to be a singular word. As such, I implemented a format for the data sending, by prepending all game words with “#” and all queries with “@” so as to differentiate between the two commands. I also added data parsing to extract the words and commands accurately. I used the highest baud rate of 115200 and managed to achieve accurate transmission between the RPi and the Pico. I simulated the entire game loop by having the RPi send a word to the Pico. (The Pico received the word usually on the first or second attempt by the RPi). Following this, I simulated the RPi querying the Pico for the word and the Pico was able to successfully transmit the word to the RPi and the RPi also changed the text on the row LCD.

  • I also created an FSM diagram to showcase the different states within my program.

  • I also asked Quinn about the PCB Milling Machine and came up with alternatives.

 

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?

  • My progress is on schedule and I have completed most of the prototyping for one block interface, except for displaying the word on the block’s LCD.

 

What deliverables do you hope to complete in the next week?

  • Next week, once the Picos arrive, I hope to experiment with the testing of 2 and more block circuits. This will involve the use of the multiplexer.
  • I also hope to help with the construction of an actual block once the acrylic blocks arrive. The block would consist of a perforated board with the connections soldered to connect the Pico to peripherals such as the LCD and pogo pins (for UART)

Wen Hui’s Status Report for 2/15

What did you personally accomplish this week on the project? Give files or photos that demonstrate your progress. Prove to the reader that you put sufficient effort into the project over the course of the week (12+ hours).
– I crafted the design requirements and made calculations to ensure that our choice of parts fit within the design requirements.
– I helped with creating the block diagram to represent the different components.


– I also helped to design concrete testing procedures in the area of unit tests and integration testing with clear quantitative metrics to measure our project’s success
– Based on our ideation, we have designed that there will be three main components in the central RPi: the WebApp, Game Controller and Embedded Controller. Since I am working on the embedded communications, I created a flow chart for each of the functions of the embedded controller based on the inputs from the Game Controller and the physical hardware peripherals


– Since we received the RPi this week, I set up the RPi, installed the relevant libraries we will be using and set up the basic test programs for UART, I2C and GPIO testing based on the libraries I researched last week. I could only test UART since the components have not yet arrived
– I also designed a simple algorithm together with a message sending protocol to handle the sending of the words between the blocks and the grid.

Is your progress on schedule or behind? If you are behind, what actions will be taken to catch up to the project schedule?
My progress is slightly behind schedule as we did not expect that the initial integrated LCD screens with ESP32 would be sold by a banned vendor. As such, we only ordered the new LCDs on Wednesday after researching for alternatives. I will work on some tasks that has to be done next week (such as planning out the circuit connections) before the parts arrive so that I can still be on schedule.

What deliverables do you hope to complete in the next week?
Next week, I have to interface the RPi with the actual components such as Pico and LCDs once the parts arrive. I would be doing individual interfacing between the RPi and each of the peripherals using UART, GPIO and I2C according to the libraries I have tried out this week. I will also start building the breadboard circuitry to establish a prototype grid + single block circuitry.

Wen Hui’s Status Report for 2/8

This week, I prepared for the proposal presentation and conducted research on answering potential QnA questions. In terms of our project, I conducted research on the different LCDs and compared their specifications.

For each of the components that we are using, I also researched about how to interface with them. We will likely be using I2C for the LCD, GPIO library for the buttons to detect a rising edge and UART to communicate with the block microcontroller. I also researched about the libraries to use for the RPi and ESP32 / Pico in terms of UART, I2C and SPI communication. I decided that we can use Python to write our embedded code on the RPi since there is a RPLCD.i2c, RPGPIO and Pyserial libraries which allows us to interace easily with the components from the RPi. Using Python would allow us to integrate easily with the web application code as well since the backend will be written in Python.

I also wrote the implementation flow for how the embedded devices would communicate with one another.

Pre-Game:

During game:

I also drafted out a top level plan in psuedocode for how the implementation would look like on the RPi in terms of the functions and variables required. The main thing I would need to test out on the RPi would be the reliability of the serial communication with the block microcontroller. Since I am trying to implement serial communication with ACK, my current idea would be for the RPi to send a message to the block microcontroller, wait a certain period of time for an ACK and repeat the sending if nothing has been received. I am not sure if this would be the most reliable way to implement the communication until I test it out.

For the block microcontroller, I did research on both Pico and ESP32 as I was not sure which we will be using yet. For ESP32, we can interface with it using Arduino IDE and it also has a local file storage which allows us to store the block’s work. For Pico, we can load Thonny OS onto it to allow it to run MicroPython. It also has a local file system to store the word from the RPi. If we do use Pico, I will have to do more research into the limitations between using C vs Python on the Pico in terms of functionality and latencies.

I am currently on schedule for my tasks. In the next week, I hope to get started on writing the code for the Rpi after finalizing the parts list and procuring the microcontroller + LCD for the blocks.