pkm-cabt-ppo — self-play agents for the Pokémon TCG (cabt engine)

Policy/value networks trained by PPO self-play to play the Pokémon Trading Card Game inside the kaggle_environments cabt engine.

Code: https://github.com/jibai-devs/pkm

Model

A pointer-style policy/value network. The engine hands the agent a variable-length list of legal options each decision, so the network cannot use a fixed action head:

• The board state is embedded into a vector h.
• Each option is embedded into o_i and scored against h (plus a summary of the options already picked this decision), giving a softmax over the variable-length option list.
• Multi-select decisions decompose into sequential picks. A learned STOP row is appended to the option list, becomes legal once minCount picks are made, and ends the sequence when chosen. A decision's log-prob is the sum of its picks'.

Sizes: card embedding 32, attack embedding 16, option-type embedding 8, option encoding 64, hidden 128. Roughly 0.5M parameters — small, because the hard part here is the action space, not capacity.

Files

File	What it is
policy.npz	numpy export of the 02_dragapult net — torch-free inference
02_dragapult/ppo_latest.pt	PPO checkpoint, 10,000 iterations (default agent)
01_psychic/ppo_latest.pt	PPO checkpoint, 1,000 iterations
00_basic/ppo_latest.pt	PPO checkpoint, 200 iterations
00_basic/exit_latest.pt	expert-iteration (IS-MCTS distillation) checkpoint
*/deck.csv	the 60-card deck each agent was trained on
*/ppo_train.csv	per-iteration training metrics

policy.npz exists because the Kaggle submission bundle has a ~197 MiB cap and torch does not fit. pkm/rl/numpy_policy.py replays the same forward pass in numpy, so the shipped agent runs on numpy alone.

The weights are deck-specific. Card identity is embedded, and each agent self-played only against its own deck — load a checkpoint with the wrong deck and it will play badly. The matching deck.csv is in each directory.

Results

Evaluation win rate against a random-legal-move agent, at the final iteration:

Agent	Iterations	Eval win rate	Games
02_dragapult	10,000	100%	20
01_psychic	1,000	95%	20
00_basic	200	~80% (as documented in-repo)	—

These are self-play agents measured against random, on a 20-game eval, against their own deck. That is a low bar and a small sample: read them as "training is working," not as a strength rating. No head-to-head vs. IS-MCTS or vs. human play is reported here.

Usage

from huggingface_hub import hf_hub_download

npz  = hf_hub_download("TomatoCream/pkm-cabt-ppo", "policy.npz")
ckpt = hf_hub_download("TomatoCream/pkm-cabt-ppo", "02_dragapult/ppo_latest.pt")
deck = hf_hub_download("TomatoCream/pkm-cabt-ppo", "02_dragapult/deck.csv")

Load the torch checkpoint with the network from the repo:

import torch
from pkm.rl.model import PolicyValueNet

model = PolicyValueNet()
model.load_state_dict(torch.load(ckpt, map_location="cpu", weights_only=True))
model.eval()

Or run torch-free, the way the Kaggle submission does, via pkm/rl/numpy_policy.py against policy.npz.

Training

PPO self-play with checkpoint-pool opponent sampling and potential-based reward shaping on the prize differential. 00_basic additionally has an expert-iteration checkpoint: IS-MCTS with determinization (for imperfect information) generates targets, which are distilled back into the network.

just train 200 16 deck/01_psychic.csv   # PPO self-play
just exit-train                         # expert iteration
just export                             # -> policy.npz

Limitations

• Deck-specific, as above; no multi-deck or opponent-pool generalization yet.
• Evaluated only against a random agent on a small sample.
• Tied to the cabt engine's observation schema; the encoder assumes it.