Author: benjamib

Working on finishing integration, making sure signals from the chip and card systems are processed correctly and progress gamestate according to the rules. I am also working on various UI improvements for increased player usability and ability to clearly see the gamestate at a glance.

This week has been mostly cleaning things up, and continuing the integration process. I have refactored some of the training code in order to more rigorously verify training improvements, and am retraining models. Otherwise, everything is going as planned.

All of the code is done, but I had to write a parallel training module so that training would occur faster by parallelizing it over multiple machines. Then I had to spin up a bunch of EC2 instances and have them train concurrently, and combine the regret weights to generate the cumulative model. The final, highest strength model is still training, and should be done by next Friday.

All infrastructure, hardware interfaces, bindings, and AI are done. AI is mostly trained, still training for the hardest difficulty. I have done significant debugging, as well as implementing more hardware integration interfaces and the entire graphics module for displaying game state on the screen.

I am working on designing the APIs to correctly target the other subsystems for easier integration. In addition, I am trying to optimize training since it is progressing quite slowly, and developing an easy way to checkpoint the system, since I currently do not have a way to save model weights and modularly swap them out for a “lower level” model.

The CFR framework and subgame solver (depth currently = 1, but any deeper is unlikely to run locally on the raspberry pi) are done. I am working on getting arbitrary depth subgames to work as well. Training has also not passed CFR test cases, so I am working on debugging that.

The ML framework itself is mostly complete. The base CFR mechanism, monte carlo simulation framework, and stochastic node selection are all done and work together. I am working on subgame solving, and will also need to review and potentially modify the APIs with which information can be added to the game in order to interface with physical components of the system.

This week I’ve begun writing the ML framework on which to train the poker AI. I’ve implemented a system to run Monte Carlo simulations, and am working on implementing stochastic node selection for training. In addition, I’ve implemented the core of the CFR portion of the algorithm which will determine the model weights.

This week I got into developing the base poker game software, and working on making sure that it is reasonably optimized to be able to run large scale simulations on it effectively. In addition, I spent a lot of time working on the design presentation for next Monday.