Author: yooraek

I did not get to do a lot of work this week because I was not in a very good condition for many days in the week. This week, I visited the lab several times to take photos of the edge cases (including when the poor quality photo, castling, and checkmate state) and test it. I added a helper to fully handle a castling move in our main file, and I updated test cases on castling for the chess game logic. There were some confusion on the coordinate since we changed our camera orientation, so I checked the coordinate conversion between function for consistency. I am planning on investing a lot more time on capstone for tomorrow and next week. I plan to go to the lab tomorrow to take more samples to test, and planning on working with everyone on finalizing integration.

This week I spent time in class editing errors in integration mostly related to coordinates. I also added a logic from python chess library that checks if ‘checkmate’ has reached to signal the user that the game has ended. I edited an error on check valid castling from chess game logic.

I also spent most of my time writing exhaustive testing for chess logic to add onto the metrics for our final presentation. I have written 90 test cases, with varying board states. For each chess piece type (6 types in total), I wrote five valid move tests, five invalid move tests, and five capturing move tests. Along the way, I am recording an error I have found, and editing the code to take account of the edge case. So far, bishop’s left downward diagonal move and bishop’s en passant capturing move had errors and I have updated the code. I am planning on adding more test cases to detect more errors I haven’t found.

I am planning on adding more test cases including invalid capturing moves to detect more edge cases I haven’t considered and helping Anoushka with testing CV tomorrow. I will also spend next week on preparing for final presentation.

This week I focused on making the CV work on our project environment. The difference detection of our code depended on absolute difference values from RGB color code, and the current black and white pieces did not show significant difference between the difference value on a square where the change has been made, and where the change has not been made. To make the absdiff value more distinct for the squares where a change has been made, I made an order for red colored chess piece set (that distinct from green and white squares) to decrease the error rate of the difference detection. I finished an algorithm for change detection with red pieces and updated it before our interim demo. Red pieces showed significantly lower error rate when detecting changes.

I also visited the lab on Thursday and Friday and took more sample photos from webcam. So far, we did not have a web cam stand and relied on photos that we took from our phone to test our CV. Since the webcam for our project produced photos with significantly lower resolution, we needed to make an update on our CV. On Saturday, I met up with Anoushka to make edits on edge detection because our CV was not detecting every vertical and horizontal lines in the photos. We made update on houghline thresholds, change detection threshold, and rectifying algorithm to make the CV work with the photos produced from webcam with lower quality.

I have also started on integration, on unifying coordinate outputs of CV and valid move logic so that it will work well with the stockfish AI. I will be focusing on translating coordinate system of our project for integration on Sunday, and will begin to work on piece identification CV algorithm when our newly ordered black chess pieces arrive. When user makes a capturing move, a color of chess piece has to be identified before updating a board state. I am planning on comparing RGB or HSV value to identify black and red pieces.

This week we finished up our individual works and began to focus on integrating our work. Demi recreated the bottom cardboards of the board and installed the LED matrix with the new LED we purchased. She connected the push button to RPi so that user can signal the move is done. Demi also set up the webcam and took sample photos to solidify our CV, and she focused on integration of our project with Stockfish AI. Anoushka also worked on AI and CV integration. So far, we did not have a webcam stand and we have been testing our CV from sample photos we took from our phones. After we set up the webcam, we ran into some issues with CV because the photos from webcam had lower resolution photos. Yoorae and Anoushka spent time on fixing the errors and debugging CV with the new sample photos from the webcam, and testing the new sample photos.

We made a purchase of red chess pieces set to solidify our change detection algorithm for CV. Yoorae wrote change detection code with red piece set.

This week I made an update on Chess Game logic to make testing easier. I added some modules that make changes on the board state, return selected positions, and checks the empty spaces, to help the testing process. I also include test cases on the test bench.

I also focused more on CV this week. I concentrated on detecting the coordinate of the board where a change has been made, while Anoushka focused on the better quality of grid lines through neural net training. The quality of initial sample images were not good enough for detecting changes, since the coordinate of the board center was inconsistent throughout the states. The previous sample photos had inconsistent grid lines, which made change detection impossible. So I took another sample photos at a lab on Wednesday in a more consistent setting with camera held in one place. I edited new sample photos through CV so that the coordinates of each square is aligned.

I also made an update on Hough line threshold Anoushka’s code that get the gridlines from edge detection so that the correct grid lines will be generated from more varying photos.  The most significant obstacle from our CV change detection is inconsistent reflection on our board. Since our physical board is made out of glassy acrylic material, anything that gets in the way between the light source and the board gets reflected on the board (such as user’s hand, camera, user’s hair and etc).

From the photo above, change detection module should output coordinate (4, 4) and (4, 6). However, no matter what method I try, (abs diff, structural similarity, background subtraction method on grey scale, RGB scale, HSV scale) threshold could not separate coordinates without changes from coordinates with changes, because the output difference value of the coordinate with different reflection on the acrylic piece was too high.

This is a huge problem in our CV because reflection on our board is going to be inconsistent throughout the whole game. I updated a working code that computes the difference with limited size of the square (at center) so that it can disregard difference from background reflection. However, this code will not work if the user places a piece away from the center of the square. The other solution will be to remake the board with an unreflecting material, such as silicon. For another solution approach, both me and Anoushka tried to blur out the reflection, but was not achievable.

Next week I will mainly focus on coming up with a solution approach to this reflection problem and to come up with a stable change detection algorithm.

This week, in a team meeting with Tamal and professor Bryon on Monday, professor has pointed out the edge cases that I have missed for the chess game logic. To update the Chess game logic to take account of every possible edge cases that are possible, I researched the chess rules in depth and I made an update on the struct of board state to return the location of each piece name and the piece color to take account of every possible edge cases in the chess game. The board state struct will also contain an int counter to take account of number of turns that have been played, to check the allowed moves for pawn. This will make the check valid process a bit slower, but it is necessary to correctly process many edge cases in the chess game such as  ‘en passant’ state of the pawns and ‘castling’ state (in which two pieces can move in one turn).

 

I also wrote test benches to check the correctness of my current structure. The current test bench consists of 50 general moves, and each special move per edge case. My logic is currently returning a 100% correctness according to the test bench results.

Lask week, our group spent most  of our time on writing our Design report. We updated our Gantt chart to make updates on changes of current schedule. Anoushka worked on introduction, design requirements, CV trade studies, and CV system description. Demi worked on Architecture, LED trade studies, and hardware system description. Yoorae helped Anoushka on design requirements, and wrote trade studies for AI engine, Project management, and ethical issues sections. We met on Thursday night last weekend to combine our work and finalize our design reports. This week, we discussed ethical issues on Monday to prepare for ethics discussion and ethics assignment for our project. We have concluded that our major goal of the project was to create leisure activity for people who enjoy chess but who are not very familiar with technology by eliminating any software interactions from user’s point of view, but can cause ethical harm by creating more social isolation since our project does not provide any user to user interaction.

 

One of the most significant risks still remains the accuracy of the CV. This is being managed by ensuring that the board does not have any qualities that would make detection harder. For example, Anoushka found that even slight irregularities on a square can cause issues. We are ensuring that the board does not have any of these. In terms of contingency plans, the main one is to ensure that the user has a way of telling us the detection is wrong. This took some planning to ensure that the game doesn’t become unplayable, but we got help from professors and TA. We decided to light up LEDs to display the detected move and provide the user with a button to press if the detection is wrong. There were no changes to the system diagram after the design report.

 

Updated Gantt Schedule:

 

Last week I have spent most of my time on lab hours for writing Design Review Report. The portion I was responsible for was any part that was related to integration of Stockfish Engine of the project. I was also responsible for writing project management portion (including schedule, team member responsibilities, budget, and risk management), related ethical issues, and related work.

I have also finalized the chess logic algorithm that returns boolean values on whether player has made a valid move or not. Since every chess piece has different allowed movements, I have created a class for each chess piece. The main function ‘validate’ will be inputed the current board state, chess piece type,  and a new location that user has indicated. The original state of selected piece (before movement) will be reachable in a board class that is passed into the function. If the movement is validated from the piece class as a possible movement, the function will return a tuple of new coordinates for the piece, else, it will return Null and the function will return false.

 

I am also working on a transition function for Chess AI integration. Anoushka has updated coordinate algorithm from CV edge detection, which has to be translated to be computed with bit boards in Stockfish engine.

From next week, I will be working with Anoushka with the  background subtraction algorithm to help her with updating board states.

This week I spent most of my time practicing for Design Review presentation. I practiced speaking with my teammates and practiced answering possible questions that can be asked to me. I also started writing Design Review Report, and wrote a code in C that can detect any illegal moves in Chess.