Category: Team Status Reports

We have integrated most of our system (Cody’s computer vision model, Denis’s MQTT and user app, and Jolie’s game state tracker and board checker). We decided to take out the main user display board as it seemed redundant considering we can just flash the validity and score to the player after the “Check” button is pressed. With less than one week left, we are looking solid and close to completion. We still have some testing to do including our user study in addition to the video, poster, and final report to finish.

 

Tests Completed:

• MQTT Data size test: We tested varying sizes of data packets to see their effect on the latency of communication. We decided to use packets of less that 300 Bytes as a result
• Individual CV letter tests: We tested the accuracy of identifying individual letters in order to see which ones were the least consistent. This led to us adding additional marks to the I and O letter tiles
• Size of Database vs accuracy and time: we tested varying database sizes and their impact on the CV accuracy, as well as how long that it took to process the images. Ran 3 tests, one with a full, a half and a minimal version of the database. The tradeoffs were as we expected, the larger the database, the more accurate but the slower it was.
• Testing scoring and validity: We tested a variety of different word locations and valid/invalid words on static boards, this led us to being certain that our sub functions worked

Last week we presented our interim demo in which we showed off the individual aspects of our project. While we found it slightly challenging to fit our work as well as a sufficient overview of our project into the constrained timeslot, we believe it overall went fairly well. As we enter the final weeks, we will focus mostly on integration and testing as well as the necessary components for our final presentations, like the slides, poster, and video.

For verification right now, we are each testing our individual subsections, with a regression model of testing. For example, with our networked devices, we began by sending the most basic of packages, and ensuring that it was being received. Then, we began working on two way communication, having device A send data to Device B, having Device B manipulate it and then sending it back to Device A. Then we would build to having multiple devices networked, with a main device operating as a broker. Now the packets contain IDs of the device that the message is addressed too, and our receiving devices interpret these packets to determine if they are for them.

In terms of validation, once we integrate all our subsystems, we plan on tracing a picture taken with the RPi Camera through our system as it gets translated into a 2D array, validates words and calculates score as well as outputting the result to the main LCD display. We will manually check along the way through print statements that the board is updated properly. Additionally, we plan on testing the system in various lighting scenarios to ensure that the ring light is sufficient for proper lighting of the board and tiles.

In our design report, we mentioned having a focus group of individuals play Scrabble with Scrabbletron so we can determine whether our updates really improve the Scrabble playing experience. We plan on running this study once we have a working system after integration.

Hardware Info:

RPI SSH cheatsheet: how to ssh into an rpi on CMU-DEVICE:

• ssh username@IP

how to reboot rpi:

• sudo shutdown -r now

MQTT communication format:

• Index 0: Sender ID
• Index 1: Receiver ID
• Index 2: Task ID
• Index 3-end: Data

 

This week, we interfaced the wireless communication and the hint logic, creating a basic setup where given a board state on the main Pi and a tile rack on the Rpi0, we can generate all of the possible words on the main Pi and send it back to the Rpi0. We also created a communication system that will standardize all of the communication that we will do between devices.

This week we made big strides with the hardware components of the project, getting 2 raspberry pi 0s to communicate with each other. We are waiting on a couple more parts to come in, which are holding us up right now. That is currently a big risk, but we are trying to mitigate that by developing as much as possible around that, including writing a thorough skeleton code for each of the RPi’s.

We forgot to put these FSM diagrams in our Design Report. We wanted to document them here to eventually edit and add to the Final Report.

 

 

As of right now, the hardware part of our project is slightly behind. We have one Raspberry Pi communicating with my laptop, which was slightly behind our goal of having two of the Raspberry Pis communicating with each other. This is just a result of a congested schedule this week and hopefully we should be back on pace by next week. Luckily, nothing is super dependent upon the hardware right now so it does not pose much risk to our overall project.

There were no changed made to the existing design of the system in the past two weeks. Shown below are some of the processed images of words on a Scrabble board that Cody used to find the contours of letters. As he noted, some of the curve-less letters are harder to pick up.

Part A (Written by Denis): Scrabble is a global board game, with over 150 million copies sold all-time. This means that most people are at least tangentially aware of what Scrabble is.  We are attempting to make our project have a global use case by using low cost materials and portable, common power sources. The only major difference that we would have to make would be adjusting the plug that we put into the outlet, as that changes depending upon where you are in the world. If we were to create this as a world-wide product, we would have to adjust the powering of our main RPi, in order that it could work overseas. Another consideration we have made would be for those who are more unfamiliar with tech. If we were to create this as a product for consumers, we would likely send it as close to being ready as possible, and with a detailed user manual for setup/calibration.

Part B (Written by Jolie): Scrabble is sold in 28 different languages across the world. Those 28 languages encompass a large portion of the world’s population. Although our project is geared towards the English version, all that would need to be done differently is to use a different letter recognition library that can interpret characters from various languages and a Scrabble dictionary in that language. Aside from that, Scrabbletron would be versatile enough to be used in any other country where one of those 28 languages is spoken because the rules of the game remain the same. Across these languages, the need for an aid to the administrative tasks of Scrabble persist, so our project will meet these goals in any language.

Part C (Written by Cody): While we have not identified any serious environmental concerns, there are a few considerations to make for a project like ours. First, our system uses power for the CV computation and LCD displays. The plant generating this power could certainly have environmental impacts, so it’s important to make our system as energy-efficient as possible by writing quality code. Secondly, we have considered the impact of the physical materials our system uses. We are introducing multiple electrical components to the standard Scrabble setup, some of which (like the displays) have batteries which could be environmentally harmful if disposed of improperly at the end of their life cycles. One marginal improvement we’ve made to limit the physical materials is the switch to wireless communication.

We gave our design presentation this week and started working with some of the hardware we plan to use, namely the Raspberry Pi. We found out the camera is not compatible with the RPi 5, so we switched to the RPi 4, which should be sufficient. Outside of setting up some of the dependencies and cloning the git repo onto the Pi, we also began familiarizing ourselves with the software libraries we plan to use, but we are open to changes based on the feedback we receive from our design presentation.

This week, we primarily focused on our design presentation. We refined our design, moving away from a wired FPGA implementation to a wireless approach. To accomplish this, we plan to use Raspberry Pi 0’s for each of the displays (one per player, plus a main display) to communicate with the primary RPi, which will handle most of the computation. MQTT and HTTP Rest APIs are the two primary approaches we are currently considering, with the former being our current planned approach.

A was written by Jolie Wasserman, B was written by Cody Breidenbach and C was written by Denis Lucey.

Part A:

It may seem like our project doesn’t directly deal with public health and welfare, but there are subtle benefits involved. First off, Scrabble is an intellectually stimulating game, so taking away the administrative tasks of the game allows players to focus more heavily on their strategy and lexiconal prowess. By adding hint capability to Scrabble, it enables novice players to have an enjoyable game experience, encouraging them to play more often, which ultimately acts as a positive impact on mind stimulation. Novice players will also feel less anxiety towards the game with the hint capability as they will be less embarrassed of their word choices compared to a more experienced opponent. Finally, each player gets their own individual touch display, so there isn’t any cross contamination of germs compared to using a main touch display. 

 

Part B:

Our social considerations included the social experience of a group of people spending time with each other while playing a board game. While web app approaches are certainly simpler while achieving the same functionality, we believe there is inherent communal value involved with the in-person Scrabble experience. Hence, we have sought to capture the convenient aspects of playing Scrabble online (word validation, automatic score tracking) while maintaining the personal aspect of the physical game. However, we have not identified significant political issues or other social concerns.

 

Part C:

Our project has a few economic considerations, especially considering how it may help new players get exposed to the game of Scrabble. Our solution as proposed would be a good companion piece to both new and experienced players, whether it’s providing new players with guardrails or allowing more experienced players to check their words against our suggestions. Because our solution will be relatively inexpensive, these groups will not be priced out. Addressing the production and distribution, because all of our components are widely produced, there are no concerns about parts being unavailable, making it easily consumable by users.

We changed our existing design from a centralized dealing mechanism to users inputting the tiles they are dealt. This way, we only need one camera for board observation. Also, we then mitigate the risk of incorrect letter recognition because we can check that new tiles recognized on the board were indeed in the player’s rack. Before, we were relying on two instances of letter recognition by our computer vision. Now, we have a method for increasing our confidence that the letter recognition was correct.