Author: cbernits

This week I worked on the CV implementation. I had taken two pictures of a chess board in different positions with my phone camera and processed those and it appears to detect correct between moves. However, because we don’t have the camera and board fully set up on our flexy board I have not been able to test. Next week, I plan to help setting up the camera and making sure the CV algorithm works fully with our project. This is slightly behind schedule, but I think if I can get this part fully working by the end of next week I can move on to our ML implementation for chess after and it will be smoother as the grounds for testing will already be in place. If we haven’t fully set up the camera system to take pictures, I’ll switch my tasks to work on the software GUIs first. I will also work on any code for the interaction between the camera and the raspberry pi next week to try to get that fully set up.

This week a large majority of my work was on the design document. I worked on Sections III and the corresponding block diagram, Sections VI, VII, VIII (including updating and making our gnatt chart more descriptive and BOM), as well as IX, references, and making smaller edits to other sections where required.

 

While this week the majority was dedicated to the design doc and the finalizing our design choices and components, I also begun working on local testing of our CV algorithm that will be used across all 3 board games. I plan to finalize this next week and then begin the ML portion for the piece classification of chess pieces.

This week in classes I did my peer reviews and listened in on the design presentations outside of class I kept considering the raspberry pi’s CV system and how it would determine the moves based on the game and it’s connection to our STM32. I was previously debating having  an initial baseline from the start of the game that will be told by the user. So they will say “chess” and that’s how the system will know what pieces are where exactly in the beginning as to avoid having to do any ML for the games we are on to have piece classifiers. However after our design presentation we had considered we realized we would need a classifier for pieces for all our games any way because there are positions like in chess here a mere diff won’t tell us exactly which piece was taken if there is more than 1 option for taking. Our CV will be Diff based from start of turn to end of turn which will be determined by a button but also have an image classified. I started working on this implementation. The Pi would communicate to the STM by sending the following to the STM:

HOME

MOVE(from_x, from_y, to_x, to_y) or probably more simply MOVE(to_x, to_y)

STOP

the STM would send back:

RECIEVED, DONE, or ERROR

 

I started considering and drawing out our states and the communication between our two controllers

This week toward the start of the week I went over some of the comments from the first presentation and discussed with my team some of the feedback. We discussed we would have probably 2 games which we want our MVP to be for which would make it easier. We then discussed hardware options that we wanted to build our frame and how they would be controlled and connect to the software. I looked for specific components and the microcontrollers we would need for each components (CV and motor control). I was sick for the second half of this week but started considering the best approach for our CV algorithm.  I considered some options of:

1. preloading and telling the algorithm what game we would be playing where it would split the board upon detection into the number of squares required (8×8 for chess) and then compute an occupancy score for each square and determine which pieces have moved

2. We take an image before and after when the user presses a button and just subtract the images, this way we should have blobs where the pieces moved and can determine what pieces are where.

3. an ML model to detect whether each square is classified or could us a YOLO model approach and detect the actual piece if this was for chess for example

4. marker on top of each piece that had a specific (red/orange etc) color and was easy for the camera to detect.

 

I did two main tasks last week. First I worked on the presentation slides and worked on finalizing those for Sunday’s due date. I then started looking into how we would implement a CV algorithm that would work for a set of boards and be able to communicate with the motors for the magnet. I plan on starting to implement this algorithm next week in actual code with a test setup board before we actually make our own. I also added rough costs for how much each component that we need are in the google drive link posted by Harrison. As this is one of the starting weeks and we had the presentations this week, I believe we are on track as long as we get the components we need and start ordering them next week so they arrive on time.