This week, I primarily worked on the character recognition software. After some research, I found it was likely the best to first run some preprocessing to convert the image to black and white, then isolate and draw onto the original image the contours of each character for better performance of the OCR. Currently, most characters are able to be recognized, but some without curves (L, T, I, etc) are proving more difficult. I plan to further tune some of the preprocessing and contour recognition to improve this. Once the individual characters are identified, I will work on the logic to build adjacent characters into words and eventually map them to the board for score calculation. Additionally, I assisted my team with the design report.
Jolie’s Status Report for 10/5
This week, I began coding some of the hint module. In the paper which we are basing the algorithm off of, they use something called cross-checks (possible letters that can fit in a word vertically if a horizontal word is played and vice versa), and I wrote a function that takes in the state of the board and updates the cross check set for each square. I also wrote a function to find anchor squares (empty squares directly adjacent to filled squares), which will be a starting place for word generation.
I got the Raspberry Pi camera set up and took a picture of our Scrabble board to start with initial testing with our cv. It was hard to get a great angle with the Pi camera so making some sort of intermediary rig might be helpful for the future. Here are the pictures I took:
Denis’s Status Report for 10/5
This week I researched the specific RPi models and LCD screens that we should use for our project. I also began looking into the MQTT protocol and how that can be implemented on the RPis. This upcoming week, I plan on beginning to interface and send data between 2 RPis as well as beginning to create an interface that we’ll be able to use on the LCD screens. My plan is to create a mock-up for what we want the screens to look like, and ensure that we can receive an input from the touch screen. We are on pace for our project.
Cody’s Status Report for 10/5
This week, I gave our design presentation. I also helped Jolie setup the RPi and spent some time figuring out how to use the camera. Additionally, I began working with some of the Pytesseract functions, and I expect image_to_data to be particularly useful, as this returns (among other things) the location of the recognized letters. I’m still not completely sure how I’ll map the words onto the slots on the board, but feedback from the design presentation may provide some insight. The idea of color recognition was proposed, although I feel this could have some difficulties, since that might require onboarding an additional CV library, and the colored grids could be covered by tiles anyway.
Team Status Report for 10/5
We gave our design presentation this week and started working with some of the hardware we plan to use, namely the Raspberry Pi. We found out the camera is not compatible with the RPi 5, so we switched to the RPi 4, which should be sufficient. Outside of setting up some of the dependencies and cloning the git repo onto the Pi, we also began familiarizing ourselves with the software libraries we plan to use, but we are open to changes based on the feedback we receive from our design presentation.
Team Status Report for 9/28
This week, we primarily focused on our design presentation. We refined our design, moving away from a wired FPGA implementation to a wireless approach. To accomplish this, we plan to use Raspberry Pi 0’s for each of the displays (one per player, plus a main display) to communicate with the primary RPi, which will handle most of the computation. MQTT and HTTP Rest APIs are the two primary approaches we are currently considering, with the former being our current planned approach.
A was written by Jolie Wasserman, B was written by Cody Breidenbach and C was written by Denis Lucey.
Part A:
It may seem like our project doesn’t directly deal with public health and welfare, but there are subtle benefits involved. First off, Scrabble is an intellectually stimulating game, so taking away the administrative tasks of the game allows players to focus more heavily on their strategy and lexiconal prowess. By adding hint capability to Scrabble, it enables novice players to have an enjoyable game experience, encouraging them to play more often, which ultimately acts as a positive impact on mind stimulation. Novice players will also feel less anxiety towards the game with the hint capability as they will be less embarrassed of their word choices compared to a more experienced opponent. Finally, each player gets their own individual touch display, so there isn’t any cross contamination of germs compared to using a main touch display.
Part B:
Our social considerations included the social experience of a group of people spending time with each other while playing a board game. While web app approaches are certainly simpler while achieving the same functionality, we believe there is inherent communal value involved with the in-person Scrabble experience. Hence, we have sought to capture the convenient aspects of playing Scrabble online (word validation, automatic score tracking) while maintaining the personal aspect of the physical game. However, we have not identified significant political issues or other social concerns.
Part C:
Our project has a few economic considerations, especially considering how it may help new players get exposed to the game of Scrabble. Our solution as proposed would be a good companion piece to both new and experienced players, whether it’s providing new players with guardrails or allowing more experienced players to check their words against our suggestions. Because our solution will be relatively inexpensive, these groups will not be priced out. Addressing the production and distribution, because all of our components are widely produced, there are no concerns about parts being unavailable, making it easily consumable by users.
Denis’s Status Report for 9/28
This week I spent some time looking into the different FPGAs that we could use for our designs. Then, after discussing with my group, we pivoted to a design without the FPGA. We all agreed that using RPi0s would enable us to do wireless communication between the main RPi and the individual screens. We also agreed that it would be easier to drive the video screens if they each had their own individual RPi driving them, as opposed to one FPGA driving all 4 screens. After changing our design, I began looking into appropriate video screens that would be able to connect to the RPi0s.
This upcoming week, I want to finalize which screens we will be using and place the order of the RPi0 wireless and the screens, so that I can start working on the wireless communication.
Jolie’s Status Report for 9/28
Throughout the course of this week, I delved deeper into the understanding of finding the best possible scrabble words given the state of the board and the player’s hand. I found an article by Andrew Appel and Guy Jacobson (from CMU SCS) written in 1988 where they implemented such algorithm. Here they used a DAWG (directed acyclic word-graph) to keep track of valid words and easily search for possible words. I plan on implementing a similar algorithm but also include functionality for searching from the anchor square looking right (the article only included an explanation of searching from left of an anchor square). An anchor square is defined as an empty square adjacent (horizontally or vertical) from a filled square. Additionally, I learned how to “cross check” newly placed letters. If a word is being placed horizontally, the cross checking occurs vertically on each newly placed tile. Each empty tile on the board is assigned a list of letters that can be placed there given the letters already placed near them. When a tile is placed, we check to see if it’s in the cross check list. If not, the move is not valid. This makes the search for possible new words shorter by taking out all references to invalid cross-check letters. Next week, I will continue to look into this method and start modularizing the algorithm.
Cody Status Report for 9/28
This week I spent some time looking to wireless communication options, since we recently modified our planned design. I have mostly been researching HTTP Rest API and MQTT as the two primary options for wireless communication. Mosquitto and Paho MQTT are the two libraries I’ve identified as useful, as MQTT is currently our first option. This week, I will be giving our design presentation and will continue to set up the software infrastructure for the project.
Jolie’s Status Report for 9/21
This week I gave the Proposal presentation for my team on Monday 9/16. Most of my weekend before the presentation was spent fixing up the presentation and practicing my delivery. Also, began looking into the most efficient algorithms for hint word generation. One way would be to make a tree-like structure and traverse through the tree with the words in the player’s rack and the words on the board. To make the problem more manageable, I will break it down to only thinking about words that can go horizontally and then tackle the vertical direction. I looked through the ECE Inventory list and requested a camera for our project based on our needs for resolution. Next week, I will continue to look into word generation algorithms for hint logic and possibly start the code if I find an algorithm that I deem is the best way out there.