Author: ziyanzho

This week I worked on system integration with my teammates. On the physical board we added frosted acrylic pieces in the holes on the board for led light diffusion and for cv to not recognize the leds through the hole as pieces.

On computer vision part everything is integrated to the system, we decided to avoid led flashing and add a button in the UI for the users to indicate they are done with a move and the leds will turn off for cv to capture the frame and then turn back on to guide the users for the next move. So there are changes made in the cv code for receiving frame capture requests.

For system integration we switched from hotspot to cmu device to connect everything. we will continue to debug and finalize the whole system especially the UI.

I am on track with schedule and next week we will be doing user testing.

This week I mainly focused on integrating the whole subsystem and with the other subsystems.

The cv part now produces results like this which can distinguish all different pieces.  The subsystem now outputs a request to the LED system for the leds to turn on and off when capturing a frame, and a FEN string to the integrate server as a result of board state detection.

I’ve also discovered some small changes that need to be made to the design of the physical board. like the need to raise the leds as close to the board as possible for the light to be seen clearly by the user.

Next week I will continue to address those small changes to improve user experience.

About new knowledge for this project, this is my first time learning about computer vision, I learned not only how to use the code for YOLO models, but also how different things would affect the outcomes of the models, how I should take the photos for the datasets to address different edge cases, how different yolo models work. About learning strategies I learned a lot by reading the official documents of yolo and anylabeling(the tool I used to label datasets), blogs about similar chess board state detection projects, and I also got help from Chatgpt when YOLOX didn’t support mps on macOS much and I had to learn to modify the code for yolox to support mps.

This week I’m mainly working on the cv part. I trained model for individual type of chess pieces, and tested them like this:

I’ve also started to integrated all the photos I took for each type of chess pieces into one model. I will need to take more photos with multiple pieces in one frame to train the model so that it will learn to tell the pieces apart. So thats my plan for next week, and also write code for the rest of the cv part such as taking a frame in live videosteam when nothing moves in 200ms, and board capture with aruco markers. Then I should be able to get output from my subsystem that can be fed into the led system and chess engine for integrating with my teammates.

I am on schedule and no changes will be made to the schedule.

About verification on my subsystem, the physical board won’t need any verification, so it’s mainly about the cv subsystem. I will evaluate detection performance of the yolox model both on precision and recall, the overall accuracy needs to exceed 95% to meet our expectations. The evaluation will be done with datasets including different circumstances such as dark environment, or different angels of camera placement. I will also measure the end-to-end detection latency from image capture to output to make sure it’s under 1s to minimize the users’ notice on latency.

the phone mount set up looks like this:

This week on physical device side I cut out the second layer of the board to hold the leds in place. I worked with Claire to make sure the design does work. For more on this part I will need a piece of design to hold this layer in place in the box, hopefully a design that will enable us to take this layer out of the box easily for led replacing. Will do during the integrating part towards end of semester.

For cv, I took videos of pieces on different positions of the board and took out 40 frame for each set for yolox training. There are 11 sets of images that I labeled. Chariot, horse, and cannon have same characters for both sides, to they each make one set, 20 images for each color. The rest of the pieces have different chinese characters for each side so they each make two sets, 40 images for each different character.  

I made the ratio of training and validation set 8:2 so 32 for training and 8 for validation.

Setting up the environment for YOLOX somehow got extremely frustrating for me, couldn’t get it to work on my mac, had to set up a whole system on a pc. A little behind on schedule, should train all datasets this week, so I need to do more work this Sunday to make sure I don’t set my team back.

This week I modified some details of the design for the board, especially the part integrating with the LEDs. The exact measurement of the distance between the individual LEDs weren’t specified, so I had to measure it myself, and with the error in measuring by hand I need to consider tolerance in the design. I decide for the top of the board to have 3 layers, on the very top is plywood layer, which is the actual board with the grid. I am deciding between 1. cutting small circles of frosted acrylic to fill in the holes of the plywood layer to better diffuse light, and 2. simply adding a whole layer of frosted acrylic under the plywood layer to diffuse light, less work more money. And then under that will be a layer to stick our LED strips on, which will be plywood strips, to reduce weight.

And I’m also taking pictures of the chess pieces to work on the cv part. I took some under room light settings, and will need to take more after working with Claire for different colored LEDs and the lamp on the phone mount we’re using.

I’m on schedule and no big design modifications were made that needs schedule changes.

This week with the cut out chess pieces and half of the board I was able to test my cv code. For the phone mount we’ve decided to buy a phone holder with lamp for holding our phone camera over the board.

Currently I cut out the chess board with plywood available in techspark which I think don’t work the best for cv because it reflects light a lot. I will try to find the other kind of plywood that doesn’t have the layer that reflects light so much. With this image you can also see that the frosted acrylic shows the background without direct light on it, which also affects cv accuracy. After I get the phone holder and lamp I will use that lamp to take photos. Since we need to implement yolox model, I will need to take like 100 photos and label them so that will be my goal next week to finishand get some yolox running.

This week on the physical chess pieces and board, I laser cut a few chess pieces out using frosted acrylic, and the top piece of the board using plywood. This is sufficient for initial testing of cv, the remaining pieces of the board can be cut out anytime after we have finalized all the physical electrical components and know the exact volume we need to hold all the parts. Also since we might need to change the design of the chess pieces later to increase board state detection accuracy, I didn’t cut out the full set of pieces, just 4 for each player for cv testing.  The parts are not with me but I’ll update images once I get a chance.

I also worked on the cv part, wrote more code for it, and will began testing next week. I am a little behind schedule on this part, as we changed our design a little last week and just got to manufacture the pieces, but I will catch up next week on this mainly.

I will also need to design the holder for the phone camera to be fixed above the board next week.