{"id":58,"date":"2023-09-23T22:24:25","date_gmt":"2023-09-23T22:24:25","guid":{"rendered":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/?p=58"},"modified":"2023-09-23T23:23:37","modified_gmt":"2023-09-24T03:23:37","slug":"nathans-status-report-for-9-23-2023","status":"publish","type":"post","link":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/2023\/09\/23\/nathans-status-report-for-9-23-2023\/","title":{"rendered":"Nathan\u2019s Status Report for 9.23.2023"},"content":{"rendered":"

This week I did a combination of research on reinforcement learning, and opponent\/platform setup to enable the RL model training.<\/span><\/p>\n

With regard to research, I want to understand as much as possible about reinforcement learning before I start the process of actually building a Mancala RL model. Through preliminary research our group decided that a self-play method of training would be best, so I read a number of papers and tutorials on both the theory of self-play RL and the logistics of putting it into practice in Python. A few of the resources I used are shown below:<\/span><\/p>\n

OpenAI Self-Play RL<\/a><\/p>\n

HuggingFace DeepRL<\/a><\/p>\n

Provable Self-Play (PMLR)<\/a><\/p>\n

Towards Data Science<\/a><\/span><\/p>\n

Python Q-Learning<\/a><\/p>\n

In order to train the self-play RL model, I must have a competent opponent for the model to start off playing against, before it can train against previous iterations of itself. If I choose too strong of a starting opponent, the model will not get enough positive reinforcement (as it will almost never win), and if I choose too weak of one the reverse is true. As such, we will start with a relatively simple minimax strategy that looks two \u201cply\u201d (single player turns) into the future. However, to build this strategy, I need a platform for the game to be played on (so the RL model can play the minimax opponent). This week I started building this platform, programming all game rules and actions, and a framework where two separate players can interact on the same board.\u00a0 I then implemented unit tests to make sure all game actions were functioning as they should. With this now in place, I have begun programming the minimax strategy itself. This means I am on schedule, and hopefully will have the minimax available to start training within the week.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"

This week I did a combination of research on reinforcement learning, and opponent\/platform setup to enable the RL model training. With regard to research, I want to understand as much as possible about reinforcement learning before I start the process of actually building a Mancala RL model. Through preliminary research our group decided that a … <\/p>\n

Continue reading “Nathan\u2019s Status Report for 9.23.2023”<\/span><\/a><\/p>\n","protected":false},"author":790,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[7],"tags":[6],"_links":{"self":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/posts\/58"}],"collection":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/users\/790"}],"replies":[{"embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/comments?post=58"}],"version-history":[{"count":5,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/posts\/58\/revisions"}],"predecessor-version":[{"id":65,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/posts\/58\/revisions\/65"}],"wp:attachment":[{"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/media?parent=58"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/categories?post=58"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/course.ece.cmu.edu\/~ece500\/projects\/f23-teama0\/wp-json\/wp\/v2\/tags?post=58"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}