AlphaZero-style Connect Four agent
Policy/value ResNet trained from zero human knowledge by self-play + MCTS (AlphaZero-style) on the standard 6x7 Connect Four board.
Play against it here: connect4-arena Space
Model
| Architecture | Conv stem -> 6 residual blocks x 96 filters -> policy head (7 logits, illegal columns masked) + value head (tanh) |
| Input | 3x6x7 planes: own stones / opponent stones / side-to-move |
| Parameters | ~1,102,248 |
| Format | model.safetensors + config.json (rebuild with az.model.create_model) |
Training
- Self-play with the latest network: batched MCTS (lockstep games, one GPU forward per tick), Dirichlet root noise (alpha=1.0, eps=0.25), tau=1 sampling for the first 10 plies.
- Replay buffer with horizontal-flip augmentation at sample time; loss = soft-target cross-entropy + value MSE, AdamW (lr 1e-3, wd 1e-4).
- Gating: a candidate replaces the published best only on >55% score over arena games vs. the current best.
- 150 iterations, 19,200 self-play games, 7.9 h wallclock.
Evaluation (final iteration, real numbers)
Elo anchored at random agent = 0; pure-MCTS-200 anchor calibrated by ladder matches at 989 Elo.
| metric | value |
|---|---|
| Elo (MLE vs anchors) | 1626 |
| vs random agent (40 games) | 100% |
| raw policy (no search) vs random | 100% |
| vs pure MCTS, 200 sims | 100% |
Usage
from az.model import create_model # az package: see the Space's bundled copy
import json, torch
from safetensors.torch import load_file
model = create_model(json.load(open("config.json")))
model.load_state_dict(load_file("model.safetensors"))
Limitations
- Connect Four is solved (first player wins with perfect play); this agent is strong but not perfect — deep tactics beyond its search budget can beat it.
- Trained only for the standard 6x7 board.
- The 40-game gating/eval matches carry ~8% sampling noise per point; the Elo curve's fine structure is noisy even though the trend is real.
- Downloads last month
- 35