What ML technique can I use for large 2d board game with a huge amount of possible states

[wide depot]

(Repost from https://www.reddit.com/r/learnmachinelearning/comments/1cb9k0m/need_help_with_creating_an_ai_for_a_2d_board_game/)

Some time ago I've made a game (http://mterczynski.pl/kulki/) where the goal is to set 5+ marbles in a line and achieve the highest score before the board is fully filled and I wanted to add an AI to it.

I could make an AI that follows an algorithm but I wanted to use Machine Learning to get better results and learn something new.
How would you approach this topic? I've made a simple Q-learning AI for Tic Tac Toe in Python, but I'm not confident that this approach would work on a 9x9 board with 6 different colors of marbles and many possible moves.

[faint palm]

Hi! I dont have much experience in this topic, but ive looked into it briefly before. Field of machine learning is called Reinforced Learning (RL). When state (observation) space becomes too big, like in chess, you might want to use neural networks to approximate this huge 'lookup table'. Just search for types of RL. For example there is a Deep Q-Networks (DQN), which might be next logic step for you.

if you want look into RL, you might like this resource:
https://gymnasium.farama.org/

Ive personally enjoyed playing around in Unity for few days
https://gymnasium.farama.org/environments/third_party_environments/#unity-ml-agents-environments-for-unity-game-engine

[crystal depot]

Nice game. You are right that tabular q-learning will struggle with this, it's a lot of states. You could try something like deep q network or PPO, or even better MCTS (bit more complex) which does a planning search to see what the best move in the position is. This tutorial is good https://huggingface.co/learn/deep-rl-course/unit0/introduction and you have a head start since you've already done the basic q-network.

[distant siren]

You could use the MU Zero version of MCTS

[wide depot]

Thanks, I found this one https://github.com/werner-duvaud/muzero-general, have you tried it?