In the past 2 weeks, I worked on many different components of the system and a large chunk of my work was on integration.
We had taken out our code from the different systems (Motor, CV, Software) and combined them into a GitHub Repo in order to proceed with further integration.
You can see the progress and commits here: https://github.com/ChrisBernitsas/FlexyBoard/commits/main/
I integrated the CV pipeline, the GUI, the Pi Bridge, and STM32 into one end to end system.
CV Changes:
I changed the CV system so that it used contours to detect groups of changed pixels rather than a straight up diff which proved more useful for lighting changes and with some fine tuning of threshold levels has proven to be very accurate (need to run tests again to obtain another number, but I do not think it has made a mistake when coupled with the game legal move handling). I also incorporated legal move handling for chess and checkers including special moves. Based on the detected squares changed there is a legal move resolved which uses those changed squares to find which move likely occurred. Accompanying this, there is player 1 move validation so weak CV detections are rejected and retried instead of continuing to player 2 in a corrupted state.
Motor Changes:
For testing: I made multiple scripts to test the accuracy in various scenarios. For example, one of these scripts took one chess piece and moved it to every single one of the 64 squares one by one lifting and removing the magnet each time and then returned to its original position to see if it was accurate enough to stay within the chess square each time (although some minor accuracy issues which we can fine tune, it was accurate enough)
Added percentages to our STM move sequencer, this allows instead of strictly a1 -> b2 or 0,0 -> 7,7, we can add percentages if we want to move pieces outside of the chessboard but within the flexyboard (for capturing pieces and other uses). For integrating with the software, I added a software move in the GUI is converted into commands which is sent to the STM which can be executed. For example:
6,7 -> 8.00%,85.71%
8.00%,85.71% -> 8.00%,82.00%
8.00%,82.00% -> 28.57%,82.00%
28.57%,82.00% -> 5,5
Added a capture area in our software which is 3 rows of 10 squares (2 at the bottom of the board where there is a lot of space and 1 at the top). These 30 squares will be used to store captured pieces and eventually be used to move for promoted pieces. The software now has an inventory which it keeps track of which pieces it moves to these captured pieces and has an order in which the 30 squares are used.
Lastly, a major part this week was the algorithm for moving pieces (especially if there are blockers like knight ‘jumping’ over other pieces). This was done with an A* recursive algorithm which turns the board into a graph with nodes and edges and computes paths given the edge weights and the blockers (if a piece blocks a path). The tie breaker for this algorithm is
1. fewer blockers (fewer pieces in the way)
2. shorter path
3. fewer turns
4. straight movement over diagonal movement when distance is equal
I had added the outer area outside the chessboard (but within the flexyboard) as part of this algorithm for path determination. So this means pieces can now move outside the chess board and back into it if there is a path that is not blocked (or if there are blockers those are moved if it is the most optimal path). Collision validation was added for both in board and out board paths. The off board inventory is treated as obstacles to move around if a piece there exists.
Promotions have not been tested yet.
ADDITIONAL QUESTION:
As you’ve designed, implemented and debugged your project, what new tools or new knowledge did you find it necessary to learn to be able to accomplish these tasks? What learning strategies did you use to acquire this new knowledge?
Over the course of the project there are many new technologies I had to learn. One of the first things was the Raspberry Pi. I had never really used a Pi extensively before. Had to learn the entire process of setting it up (flashing the SD card with the imager), setting up SSH, installing dependencies, etc… For this, one of the biggest things that helped was when I was eating lunch with my friend and asked him questions about Pis (he had used them extensively before) for like 30 minutes. Was able to ask him questions from time to time after that as well, but the majority after set up was straight forward and trial and error. I had never used an accessory connected to a Pi either obviously so had to go through the process of connecting the camera to the pi and watching a tutorial on how to actually get images and process them. I had read documentation as well about this and this helped a lot for the initial small code in the beginning when I was making sure everything was set up properly.
Communication between the Pi and STM was also something I had to learn. I had taken embedded and was pretty confident working with microcontrollers, but having to format movement commands and send them in a parseable format to the STM was something that I read on the raspberry pi forums combined with old Embedded notes and code that I had done.
There were many other parts; however, one of the bigger parts was the A* algorithm. I had been trying for a while to implement pretty rudimentary methods but nothing seemed to be working. Eventually, I decided to build something up from 15-281. I had forgotten a lot of this but was able to relearn it. In order to do this it was a lot of internet, I had watched a lot of YouTube videos to figure out the best approach I wanted and what would actually suit my requirements and ended up reading a few articles I think GeeksForGeeks proved very useful.
In general another part which I had to learn was integrating and testing. There wasn’t really any external source for this that helped. This was just a lot of manual understanding and planning. I think the learning strategy for this part would be classified as learning by trial and error, for each time a subsystem didn’t work as expected or wasn’t working with another subsystem something had to be changed. So definitely learned a lot in terms of making systems work together and integrating them together after they’ve been built individually.