Tag: status report.

For this week I worked on a few final things. I added improvements for the motion planning algorithm to allow it to move better between off board and in board, I added a check for when the manual captured pieces are put on the game board, the next turn it moves the piece to the captured area by automatically detecting where it is. Finally, I improved the automatic detection in order to revert back to this and have manual human confirmation, will make sure the accuracy is high enough tomorrow. Finally, added checks for our parcheesi implementation and made sure it could function properly and move accurately physically on the board.

This week I also updated the algorithm to detect shorter paths instead of minimizing blockers, here is an update on Motion Planner:

Parcheesi Region detection works but is not perfectly consistent with detection and with moving the pieces especially because of the strength of the magnets relative to the piece weight. May need to focus on validation and making sure our other systems and chess/checkers gameplay is smooth rather than trying to make all 3 work perfectly.

CV Detection:

In the past 2 weeks, I worked on many different components of the system and a large chunk of my work was on integration.

We had taken out our code from the different systems (Motor, CV, Software) and combined them into a GitHub Repo in order to proceed with further integration.

You can see the progress and commits here: https://github.com/ChrisBernitsas/FlexyBoard/commits/main/

I integrated the CV pipeline, the GUI, the Pi Bridge, and STM32 into one end to end system.

CV Changes:

I changed the CV system so that it used contours to detect groups of changed pixels rather than a straight up diff which proved more useful for lighting changes and with some fine tuning of threshold levels has proven to be very accurate (need to run tests again to obtain another number, but I do not think it has made a mistake when coupled with the game legal move handling). I also incorporated legal move handling for chess and checkers including special moves. Based on the detected squares changed there is a legal move resolved which uses those changed squares to find which move likely occurred. Accompanying this, there is player 1 move validation so weak CV detections are rejected and retried instead of continuing to player 2 in a corrupted state.

Motor Changes:

For testing: I made multiple scripts to test the accuracy in various scenarios. For example, one of these scripts took one chess piece and moved it to every single one of the 64 squares one by one lifting and removing the magnet each time and then returned to its original position to see if it was accurate enough to stay within the chess square each time (although some minor accuracy issues which we can fine tune, it was accurate enough)

Added percentages to our STM move sequencer, this allows instead of strictly a1 -> b2 or 0,0 -> 7,7, we can add percentages if we want to move pieces outside of the chessboard but within the flexyboard (for capturing pieces and other uses). For integrating with the software, I added a software move in the GUI is converted into commands which is sent to the STM which can be executed. For example:

6,7 -> 8.00%,85.71%
8.00%,85.71% -> 8.00%,82.00%
8.00%,82.00% -> 28.57%,82.00%
28.57%,82.00% -> 5,5

Added a capture area in our software which is 3 rows of 10 squares (2 at the bottom of the board where there is a lot of space and 1 at the top). These 30 squares will be used to store captured pieces and eventually be used to move for promoted pieces. The software now has an inventory which it keeps track of which pieces it moves to these captured pieces and has an order in which the 30 squares are used.

Lastly, a major part this week was the algorithm for moving pieces (especially if there are blockers like knight ‘jumping’ over other pieces). This was done with an A* recursive algorithm which turns the board into a graph with nodes and edges and computes paths given the edge weights and the blockers (if a piece blocks a path). The tie breaker for this algorithm is

1. fewer blockers (fewer pieces in the way)

2. shorter path

3. fewer turns

4. straight movement over diagonal movement when distance is equal

I had added the outer area outside the chessboard (but within the flexyboard) as part of this algorithm for path determination. So this means pieces can now move outside the chess board and back into it if there is a path that is not blocked (or if there are blockers those are moved if it is the most optimal path).  Collision validation was added for both in board and out board paths. The off board inventory is treated as obstacles to move around if a piece there exists.

Promotions have not been tested yet.

ADDITIONAL QUESTION:

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Over the course of the project there are many new technologies I had to learn. One of the first things was the Raspberry Pi. I had never really used a Pi extensively before. Had to learn the entire process of setting it up (flashing the SD card with the imager), setting up SSH, installing dependencies, etc… For this, one of the biggest things that helped was when I was eating lunch with my friend and asked him questions about Pis (he had used them extensively before) for like 30 minutes. Was able to ask him questions from time to time after that as well, but the majority after set up was straight forward and trial and error. I had never used an accessory connected to a Pi either obviously so had to go through the process of connecting the camera to the pi and watching a tutorial on how to actually get images and process them. I had read documentation as well about this and this helped a lot for the initial small code in the beginning when I was making sure everything was set up properly.

Communication between the Pi and STM was also something I had to learn. I had taken embedded and was pretty confident working with microcontrollers, but having to format movement commands and send them in a parseable format to the STM was something that I read on the raspberry pi forums combined with old Embedded notes and code that I had done.

There were many other parts; however, one of the bigger parts was the A* algorithm. I had been trying for a while to implement pretty rudimentary methods but nothing seemed to be working. Eventually, I decided to build something up from 15-281. I had forgotten a lot of this but was able to relearn it. In order to do this it was a lot of internet, I had watched a lot of YouTube videos to figure out the best approach I wanted and what would actually suit my requirements and ended up reading a few articles I think GeeksForGeeks proved very useful.

In general another part which I had to learn was integrating and testing. There wasn’t really any external source for this that helped. This was just a lot of manual understanding and planning. I think the learning strategy for this part would be classified as learning by trial and error, for each time a subsystem didn’t work as expected or wasn’t working with another subsystem something had to be changed. So definitely learned a lot in terms of making systems work together and integrating them together after they’ve been built individually.

This week, a major part of it was making sure parts worked together and getting ready for the Demos. I made some improvements to the CV system which had included splitting detection into multiple parts. There is now detection after the camera captures an image on the Pi. This includes a region identification for the entire Flexyboard which shows up on the image in a green grid, there is also detection for the subpart which shows up in a yellow grid (and also lays out the 64 squares). Upon improving this system this week, I had also worked on the communication between the Pi and the STM32. The Pi can now send commands to the STM32 as well as move coordinates such as 0,0 -> 7,7 (this system would be useful for chess and checkers but we would need a more precise variant for parcheesi). The upcoming week I plan to move the CV instead of using the 64 squares for chess and checker to feed output move data detection by where it is proportional to the overall board, this will make this system work better for parcheesi. I will also plan by next week to implement the software algorithm to detect if pieces need to be moved out of the way before making a piece move and then moved back. I plan to do this by detecting how many squares have pieces in the way and calculating the minimal amount of pieces that need to be displaced for a piece to make its move (it could be blocked).

Here is an image of the current output:

Extra Question:

In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

For the verification part of this, I am focusing on the computer vision (CV) and board state detection subsystem, along with the Pi to STM32 move-command interface. The main requirement for my subsystem is that it correctly detects board state changes and produces accurate move outputs reliably enough for the full FlexyBoard system to function. In our design requirements, one of the main targets is at least 95% correct move detection accuracy, so my verification testing is centered around measuring whether that threshold is met.

To verify this, I will run repeated trials in checkers first, since that game is currently the most functional, and then expand to chess and Parcheesi as the detection method becomes more generalized. For checkers, I will play full games while recording, for every turn:

• the true move made on the board
• the move output predicted by the CV system
• whether the detected move was correct
• whether the board region and square grid were identified correctly
• any failure type, such as missed detection, incorrect source square, incorrect destination square, or piece visibility issue

From these trials, I will compute the following metrics to help quantify the system’s abilities:

• move detection accuracy = number of correct detected moves / total moves tested
• source-square accuracy
• destination-square accuracy
• board localization success rate
• failure frequency by category

I will run enough total moves across multiple games to create a meaningful sample size rather than relying on one short trial. I will compare the final measured accuracy against the 95% design requirement. If the measured value is below that threshold, I will use the logged failure categories to determine the main causes of error, such as poor contrast between black pieces and dark squares, imperfect region detection, or square-mapping issues.

For the Pi to the STM32 communication portion, I will verify that coordinate commands sent by the Pi are received correctly by the STM32 by testing known coordinate pairs and confirming that the expected command is executed. I will record whether each transmitted move matches the commanded move and compute a command transmission success rate. This will verify that the communication subsystem is working reliably enough for integration into the larger project.

These measured results will be analyzed directly against the design requirements which we laid out int he design doc: if move detection accuracy is at least 95%, region detection is stable, and coordinate commands are transmitted correctly and consistently, then my subsystem can be considered verified for its intended role in the project.