---
license: mit
library_name: safetensors
tags:
  - reinforcement-learning
  - ppo
  - openfront
  - game-ai
  - pytorch
language: []
pipeline_tag: reinforcement-learning
---

# openfront-rl

PPO policy checkpoints for a self-play agent on
[OpenFront.io](https://openfront.io), trained with the Rust `oftrain` stack in
[djmango/openfront-ai](https://github.com/djmango/openfront-ai).

Encoders used by these policies live in a sibling repo:
[`djmango/openfront-tile-autoencoder`](https://huggingface.co/djmango/openfront-tile-autoencoder).

## Latest (use this)

| Field | Value |
|-------|-------|
| **Run** | `ppo_v11` |
| **Pointer** | [`ppo_v11/latest.safetensors`](ppo_v11/latest.safetensors) + [`ppo_v11/latest.state.json`](ppo_v11/latest.state.json) + [`ppo_v11/manifest.json`](ppo_v11/manifest.json) |
| **Update** | **2486** (milestone file `policy_update2485.*` is the last numbered snapshot; `latest` is one step ahead) |
| **Curriculum stage** | 23 |
| **Approx. env steps** | 16.2M |
| **Reward / schedule** | `v10-anti-spiral-v1` / curriculum `v10` |
| **Weights size** | ~154 MB (safetensors) |

Download the current policy:

```bash
huggingface-cli download djmango/openfront-rl \
  ppo_v11/latest.safetensors \
  ppo_v11/latest.state.json \
  ppo_v11/manifest.json \
  --local-dir ./ppo_v11
```

Or from the trainer / play scripts (they already default to this repo):

```bash
# restore into a checkpoint dir (validates manifest)
uv run python scripts/hf_checkpoint_sync.py \
  --checkpoint-dir rust/checkpoints/ppo_v11 \
  --run-prefix ppo_v11 \
  --restore-latest

# live play helper
RUN_NAME=ppo_v11 bash scripts/play_live.sh
```

Always prefer **`ppo_v11/latest.*`** over digging through numbered milestones.
`manifest.json` is the compatibility gate (`format=oftrain-safetensors`).

## Layout

```
ppo_v11/                         # current training run
  latest.safetensors             # ← production pointer
  latest.state.json
  manifest.json                  # architecture + AE refs + update/stage
  policy_updateNNNN.*            # thinned milestone history
  curriculum_advance_*.*         # snapshots at stage promotions
  curriculum_demote_*.*          # snapshots at stage demotions

ppo_v10/ … ppo_v81/              # prior runs: latest.* + manifest only
```

Older dense `policy_update*` histories (every ~5 updates) were pruned —
adjacent milestones are nearly identical for most uses. Kept history for
`ppo_v11` is every 100 updates, plus every 25 in the most recent ~200 updates,
plus all curriculum advance/demote snapshots.

## Architecture (from `manifest.json`)

- **Policy:** `oftrain-policy` schema v3 — spatial grid tower + player/unit
  streams, legality-masked discrete actions (full OpenFront intent surface).
- **Recurrent:** LSTM (`hidden_size=512`, BPTT 24 / rollout 48),
  `action-outcome-v1` context, reset on `episode_done`.
- **Observation:** frozen tile autoencoders (`ae_v32_nostatic` fine 1/8 +
  coarse 1/16) with exact-state bypass (diplomacy, scalars, transients).
- **Training:** PPO + GAE, win-gated multi-map curriculum, native engine
  (Node hedge optional).

See the living design notes in the git repo:
[`DESIGN.md`](https://github.com/djmango/openfront-ai/blob/master/DESIGN.md).

## Prior runs

| Run | Role | What remains here |
|-----|------|-------------------|
| `ppo_v11` | **Current** | latest + thinned milestones + curriculum snapshots |
| `ppo_v10` | Previous mainline | `latest.*` + `manifest.json` only |
| `ppo_v9`, `ppo_v86`…`ppo_v81` | Lineage / ablations | `latest.*` + `manifest.json` only |
| Early `ppo_v*` / `bc_*` | Legacy | removed in the HF cleanup (recoverable from git LFS/history only if re-uploaded) |

## Retention

Going forward, training pods should not re-flood the Hub with every-5-update
milestones. Use `scripts/hf_prune_openfront_rl.py` after large syncs, and prefer
syncing `latest.*` / `manifest.json` / curriculum markers plus sparse
milestones.

## License / credit

MIT, matching [djmango/openfront-ai](https://github.com/djmango/openfront-ai).
OpenFront itself is a separate project — this repo only hosts learned weights
and trainer metadata.