Posted on

Team Status Report for 4/25/26

This week we focused on finishing our remaining tasks and our final testing/integration testing. We added a physical button instead of the purely software key. We finished adding a UI for parcheesi and the appropriate on board implementation. We improving our move planning algo and detection for the board. Cleaned up board swapping between chess/checkers and parcheesi for our demo.

Until the demo we still need to fine tune a few things mainly visual in terms of the board.

Tests:

Motors:

Move Completion time: 4.42

Placement accuracy across all 3 games: (100%)

Note/Change: This number was initially more inaccurate but after tuning down the speed and slowing down the magnet pick up and let go we were able to increase our accuracy.

Magnet Pickup: 100% succesful

 

CV System (Pi + Camera):

Diff Accuracy alone: 82% (detection of extra or too few squares means a wrong detection)

Full CV Accuracy with game rules and board state: 100%

ML Accuracy: 62% (initially)

Promotion Accuracy and Detection: 100%

Change: Our low ML Accuracy caused us to switch from a trained model to a system that stored pieces on the side of the chessboard and remembered them. This meant that for promotions it could take from the used pieces. Could also detect when player 1 promotes to one of these pieces and takes it.

Auto detection of Outer board: 75%

Auto detection of Inner Board if Outer board correct: 70%

Change: Made the user manually pick the corner points in software. Coupled with changing the outer board corners to a unique color for auto detection and then using fixed geometry for inner board and then manual user confirmation.

 

Player 2 GUI:

Ran tests for invalid move detection on all 3 games: 100%

Ran tests for correctly updating given the move from the Pi:

For Chess: 100%

For Checkers: 100%

For Parcheesi: 100%

Integration Tests:

Ran through 10 full games of Chess, Checkers, and Parcheesi.

Ensured no build up in error, ensured software state matched physical state, ensured accuracy throughout the game: 100%

Session reliability: 5 game of each type was played back to back to back to ensure we didn’t need to recalibrate (for at least an hour). Total duration took 4 hours.

User Enjoyment: Ran study to ensure > 80% like it more than strictly virtual We received 100% enjoyment. (Tested on 10 people)

 

Posted on

Chris’s Status Report for 4/25/26

For this week I worked on a few final things. I added improvements for the motion planning algorithm to allow it to move better between off board and in board, I added a check for when the manual captured pieces are put on the game board, the next turn it moves the piece to the captured area by automatically detecting where it is. Finally, I improved the automatic detection in order to revert back to this and have manual human confirmation, will make sure the accuracy is high enough tomorrow. Finally, added checks for our parcheesi implementation and made sure it could function properly and move accurately physically on the board.

This week I also updated the algorithm to detect shorter paths instead of minimizing blockers, here is an update on Motion Planner:

Parcheesi Region detection works but is not perfectly consistent with detection and with moving the pieces especially because of the strength of the magnets relative to the piece weight. May need to focus on validation and making sure our other systems and chess/checkers gameplay is smooth rather than trying to make all 3 work perfectly.

CV Detection:

Posted on

Team’s status report for 4/18/26

This week there was a lot of integration that went on between the systems. Here is our combined repo code:

https://github.com/ChrisBernitsas/FlexyBoard

In terms of progress we:

integrated the CV, GUI, Pi bridge, and STM32 into one end-to-end system.

Added off board capture spaces and legal move validation for chess/checkers

Added A* based movement around pieces

Kept improving Software GUI

Kept fine tuning motor accuracy and tuning how it took inputs from the Pi (works on percentage system, square system, or coordinate system now) This makes it robust for more games.

Moved to contour-based CV detection. When testing the outer board and chessboard detection part seemed to be not up to our accuracy standards so currently it’s a manual selection at the start of the game and will be reverted back to automatic this week when we can get a unique color to outline them.

Did unit/integration testing between and for subsystems based on our design report and have numbers for some systems.

Our next tasks are as follows:

1. Fix magnets on Chess pieces/Finish Checkers pieces (I think 3 missing magnets and some are flipped polarity from the gantry magnets)

2. Improve detection of outer board and inner chess/checkers/parcheesi board unless we stick with manual.

3. Improve our system so we can slide boards in and out for fast change. (Velcro system).

4. Incorporate button for ending turn instead of enter key

5. Allow software to, at any time, update the game board with any new position/promotion

6. Promotion testing from captured area/ML

6. Confirm and fully test Algo for moving through obstacles and off board

7. Finish updating parcheesi UI in software (and all features/rules) and label squares virtually so that they can be used when sending commands.

8. Test accuracy of motor control coupled with CV system and algo for full games of parcheesi/chess/checkers especially

9. Potentially automate camera detection of if its chess/checkers/parcheesi board so don’t have to manually input (will automatically know which rules and potentially GUI to use) instead of manual selection/input

Posted on

Chris’s Status Report for 4/18/26

In the past 2 weeks, I worked on many different components of the system and a large chunk of my work was on integration.

We had taken out our code from the different systems (Motor, CV, Software) and combined them into a GitHub Repo in order to proceed with further integration.

You can see the progress and commits here: https://github.com/ChrisBernitsas/FlexyBoard/commits/main/

I integrated the CV pipeline, the GUI, the Pi Bridge, and STM32 into one end to end system.

CV Changes:

I changed the CV system so that it used contours to detect groups of changed pixels rather than a straight up diff which proved more useful for lighting changes and with some fine tuning of threshold levels has proven to be very accurate (need to run tests again to obtain another number, but I do not think it has made a mistake when coupled with the game legal move handling). I also incorporated legal move handling for chess and checkers including special moves. Based on the detected squares changed there is a legal move resolved which uses those changed squares to find which move likely occurred. Accompanying this, there is player 1 move validation so weak CV detections are rejected and retried instead of continuing to player 2 in a corrupted state.

Motor Changes:

For testing: I made multiple scripts to test the accuracy in various scenarios. For example, one of these scripts took one chess piece and moved it to every single one of the 64 squares one by one lifting and removing the magnet each time and then returned to its original position to see if it was accurate enough to stay within the chess square each time (although some minor accuracy issues which we can fine tune, it was accurate enough)

Added percentages to our STM move sequencer, this allows instead of strictly a1 -> b2 or 0,0 -> 7,7, we can add percentages if we want to move pieces outside of the chessboard but within the flexyboard (for capturing pieces and other uses). For integrating with the software, I added a software move in the GUI is converted into commands which is sent to the STM which can be executed. For example:

6,7 -> 8.00%,85.71%
8.00%,85.71% -> 8.00%,82.00%
8.00%,82.00% -> 28.57%,82.00%
28.57%,82.00% -> 5,5

Added a capture area in our software which is 3 rows of 10 squares (2 at the bottom of the board where there is a lot of space and 1 at the top). These 30 squares will be used to store captured pieces and eventually be used to move for promoted pieces. The software now has an inventory which it keeps track of which pieces it moves to these captured pieces and has an order in which the 30 squares are used.

Lastly, a major part this week was the algorithm for moving pieces (especially if there are blockers like knight ‘jumping’ over other pieces). This was done with an A* recursive algorithm which turns the board into a graph with nodes and edges and computes paths given the edge weights and the blockers (if a piece blocks a path). The tie breaker for this algorithm is

1. fewer blockers (fewer pieces in the way)

2. shorter path

3. fewer turns

4. straight movement over diagonal movement when distance is equal

I had added the outer area outside the chessboard (but within the flexyboard) as part of this algorithm for path determination. So this means pieces can now move outside the chess board and back into it if there is a path that is not blocked (or if there are blockers those are moved if it is the most optimal path).  Collision validation was added for both in board and out board paths. The off board inventory is treated as obstacles to move around if a piece there exists.

Promotions have not been tested yet.

 

 

 

ADDITIONAL QUESTION:

As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?

Over the course of the project there are many new technologies I had to learn. One of the first things was the Raspberry Pi. I had never really used a Pi extensively before. Had to learn the entire process of setting it up (flashing the SD card with the imager), setting up SSH, installing dependencies, etc… For this, one of the biggest things that helped was when I was eating lunch with my friend and asked him questions about Pis (he had used them extensively before) for like 30 minutes. Was able to ask him questions from time to time after that as well, but the majority after set up was straight forward and trial and error. I had never used an accessory connected to a Pi either obviously so had to go through the process of connecting the camera to the pi and watching a tutorial on how to actually get images and process them. I had read documentation as well about this and this helped a lot for the initial small code in the beginning when I was making sure everything was set up properly.

Communication between the Pi and STM was also something I had to learn. I had taken embedded and was pretty confident working with microcontrollers, but having to format movement commands and send them in a parseable format to the STM was something that I read on the raspberry pi forums combined with old Embedded notes and code that I had done.

There were many other parts; however, one of the bigger parts was the A* algorithm. I had been trying for a while to implement pretty rudimentary methods but nothing seemed to be working. Eventually, I decided to build something up from 15-281. I had forgotten a lot of this but was able to relearn it. In order to do this it was a lot of internet, I had watched a lot of YouTube videos to figure out the best approach I wanted and what would actually suit my requirements and ended up reading a few articles I think GeeksForGeeks proved very useful.

In general another part which I had to learn was integrating and testing. There wasn’t really any external source for this that helped. This was just a lot of manual understanding and planning. I think the learning strategy for this part would be classified as learning by trial and error, for each time a subsystem didn’t work as expected or wasn’t working with another subsystem something had to be changed. So definitely learned a lot in terms of making systems work together and integrating them together after they’ve been built individually.

Posted on

Team’s status report for 4/4/26

As a team we focused on preparing and having everything ready for the Interim  demos this week. We each prepared one piece of the overall system and got to show some parts of it off at demo specifically for checkers. In the upcoming weeks we plan on implementing player 2 software, the algorithm for moving pieces out of the way and captures, and the communication between Physical board -> Pi -> player 2  and player 2-> Pi -> STM32. (the pi -> STM32 part is already done). We realized that for some boards the CV may struggle with the black checkers and chess pieces on the board if the squares are also black so we may order boards with different colors.

Extra Question:

In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

Validation:

For the validation part of this, once we have player 2 and the connections between the microcontrollers to player 2, we will validate that the entire system works by first testing the system on one move. We will make the first move physically on the board and see if the CV system output is correctly passed to player 2 in software. If this is updated properly we will make the player 2 move in software and see if this output is correctly passed to the Pi and then to the STM and then made correctly on the physical board. If this full cycle works for one move, we will play a full game (chess, checkers, then parcheesi) to see if it works on all games and the consistency and note any flaws and correct them.  A successful validation trial means that this full cycle completes without manual intervention and that the final physical board state matches the software state. If this holds throughout each move and throughout the game we can say our design is validated. During this process we won’t just look at if it works but also all failure types. This includes the above aforementioned:

  • physical board state matches software
  • no manual intervention
  • works across supported games
  • acceptable accuracy in terms of where the pieces are and reliability throughout the whole session
  • incorrect CV detection, incorrect software update, incorrect communication between components, incorrect physical piece execution
  • loss of consistency over multiple repeated turns (if pieces get slightly farther from their correct location and error builds)

To analyze the measured results, we will record for each turn (1) whether the move was completed successfully, (2) whether the physical board state matched the software state, (3) whether any manual intervention was needed, (4) and what type of failure occurred if the turn failed.

From this, we can calculate a

  • turn success rate,
  • game completion rate,
  • and consistency rate over repeated turns.

We will compare these results against our use case requirements by checking whether the system can reliably complete moves and full games across supported games without manual correction and without accumulated positional error. If repeated trials show that the board state remains synchronized with the software state and the system can complete games consistently, then the overall design will be considered validated.

Posted on

Chris’s Status Report for 4/4/26

This week, a major part of it was making sure parts worked together and getting ready for the Demos. I made some improvements to the CV system which had included splitting detection into multiple parts. There is now detection after the camera captures an image on the Pi. This includes a region identification for the entire Flexyboard which shows up on the image in a green grid, there is also detection for the subpart which shows up in a yellow grid (and also lays out the 64 squares). Upon improving this system this week, I had also worked on the communication between the Pi and the STM32. The Pi can now send commands to the STM32 as well as move coordinates such as 0,0 -> 7,7 (this system would be useful for chess and checkers but we would need a more precise variant for parcheesi). The upcoming week I plan to move the CV instead of using the 64 squares for chess and checker to feed output move data detection by where it is proportional to the overall board, this will make this system work better for parcheesi. I will also plan by next week to implement the software algorithm to detect if pieces need to be moved out of the way before making a piece move and then moved back. I plan to do this by detecting how many squares have pieces in the way and calculating the minimal amount of pieces that need to be displaced for a piece to make its move (it could be blocked).

Here is an image of the current output:

Extra Question:

In particular, how will you analyze the anticipated measured results to verify your contribution to the project meets the engineering design requirements or the use case requirements?

For the verification part of this, I am focusing on the computer vision (CV) and board state detection subsystem, along with the Pi to STM32 move-command interface. The main requirement for my subsystem is that it correctly detects board state changes and produces accurate move outputs reliably enough for the full FlexyBoard system to function. In our design requirements, one of the main targets is at least 95% correct move detection accuracy, so my verification testing is centered around measuring whether that threshold is met.

To verify this, I will run repeated trials in checkers first, since that game is currently the most functional, and then expand to chess and Parcheesi as the detection method becomes more generalized. For checkers, I will play full games while recording, for every turn:

  • the true move made on the board
  • the move output predicted by the CV system
  • whether the detected move was correct
  • whether the board region and square grid were identified correctly
  • any failure type, such as missed detection, incorrect source square, incorrect destination square, or piece visibility issue

From these trials, I will compute the following metrics to help quantify the system’s abilities:

  • move detection accuracy = number of correct detected moves / total moves tested
  • source-square accuracy
  • destination-square accuracy
  • board localization success rate
  • failure frequency by category

I will run enough total moves across multiple games to create a meaningful sample size rather than relying on one short trial. I will compare the final measured accuracy against the 95% design requirement. If the measured value is below that threshold, I will use the logged failure categories to determine the main causes of error, such as poor contrast between black pieces and dark squares, imperfect region detection, or square-mapping issues.

For the Pi to the STM32 communication portion, I will verify that coordinate commands sent by the Pi are received correctly by the STM32 by testing known coordinate pairs and confirming that the expected command is executed. I will record whether each transmitted move matches the commanded move and compute a command transmission success rate. This will verify that the communication subsystem is working reliably enough for integration into the larger project.

These measured results will be analyzed directly against the design requirements which we laid out int he design doc: if move detection accuracy is at least 95%, region detection is stable, and coordinate commands are transmitted correctly and consistently, then my subsystem can be considered verified for its intended role in the project.

 

Posted on

Team Status Report for 3/28/26

As a team, we finished developing the arm for the camera for the raspberry pi and properly got measurements for it which we then made an arm made out of wood and attached the Camera too. All the hardware aspects should be done now and we are making progress on the CV and control components with the motors. We plan on demoing our project with checkers and plan to have the CV and motor part and communication between the pi and STM. The player 2 software and ML piece classifier for chess will probably not be fully done and will have to be completed after the demo.

Posted on

Chris’s status report for 3/28/26

This week there were a lot of things that happened. We finished testing with one motor which was quite operational. There was some choppiness which we were able to sort out. However when I connected all 4 motors one of the motors seemed to stop functioning well. I am not fully sure if something was shorted or if it’s a current/resistance issue. Will look more into this over the next couple days. Additionally, there was progress made with the CV detection. There were issues with properly identifying the corners of the board as we were still adjusting the size of the board and issues with identifying the pieces within the board. There was progress made in this domain but still not perfect. I set up a communication layer and state machine for the interaction between pi and STM32 although it is not perfect yet as I am still trying to work on getting the CV detection proper and functioning well.

 

CV Update:

Posted on

Team Status Report for 3/21/26

During this week as a team we worked on the ethics assignment and had out ethics lecture on Wednesday. We got to thinking about our product in the real world and how it affects stakeholders. We will keep thinking about this as time goes on.

As a team we have an order in place for remaining parts which include the game boards and 200+ magnets which we will need and can hopefully use soon.

We are slightly behind in some tasks but ahead in others. Overall we are on pace and plan to spend next week getting the full movement system working between the motors, raspberry pi, and STM32.

Posted on

Chris’s status report for 3/21/26

This week I was able to connect to the raspberry pi and get some preliminary code down for motor control, the camera has still not been set up and so the task of making the CV work on the actual physical system is still in play. Also the ML algorithm is pushed back a little now. I do not think this is a major issue as I started working on other parts in later weeks now due to the unset up hardware. I also started planning and working on the 3 boards for the software player who will interact with the hardware. Next week I want to have the CV fully down and working and have most of the ML algorithm done so that it will be functional in checkers and Sorry but not necessarily Chess yet. The week after will be finishing motor control and getting the communication between STM and raspberry pi set up.