This week I spent most of my time working on using CV on actual images of the chessboard that Demi made. I ran into some issues initially due to us not being able to hold the camera steady and straight when taking pictures of the board. The webcam stand will help with some of these problems, but I decided to use a neural network to refine the images so I can test even with images that may not be very good. This also gives us some flexibility with images.
The neural network I used is https://github.com/maciejczyzewski/neural-chessboard. This open-source model takes in chess board images, cleans them up and provides a zoomed in and focused image of the board. I ran into troubles with installation and use initially because the tensorflow and Keras versions were not compatible. I was able to solve this by using a Conda virtual environment and directly installing the requirements of the model.
Since our images weren’t good (tilted, camera moved between images etc), I had to train for 60 epochs to get decent results. I varied my num_epochs between 40 and 70 to find the optimum number for our images. There was not much improvement after 60, so I settled on that value. A sample result is below:
Once this was done, I was finally able to get to testing. This week I tested division of the chessboard into squares and extraction of pieces since this is the most error prone part. All squares were detected properly for 20 trials. Examples below:
2.
The next task is testing how well the code can figure out the difference between squares at t-1 and t. The image below shows a sample of the concatenated board at t=0 and t=1. The issue I am running into here is that neither structural similarity nor mean squared error is returning a steady estimate of which squares had changes. This could be due to the movement of the camera in between. I am combating this by using Background subtraction method so that changes in the background are not counted as “structural dissimilarity”.
I plan on working with Yoorae on this for the next 1-2 days. After that, I will begin working more on integration with Yoorae’s chess logic and ensuring I provide her the right inputs. 
I am on track with what I wanted to do this week and am hopefully able to resolve the hurdle with square change detection by the beginning of next week with Yoorae. For next week, my deliverable is more test metrics for the actual square change detection. I will also work with Yoorae on integrating her parts with mine.