Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,59 @@
+---
+license: mit
+library_name: pytorch
+tags:
+- reinforcement-learning
+- game-ai
+- slay-the-spire
+- reinforce
+- policy
+---
+
+# sts-rl-agent — a learned non-combat policy for Slay the Spire
+
+To our knowledge, the **first working learned policy published for [sts_lightspeed](https://github.com/gamerpuppy/sts_lightspeed)** — the fast C++ Slay the Spire simulator, which ships a 412-dim neural-network observation interface (`NNInterface`) but no trained weights.
+
+A small MLP (~100k params, `[128,128]`, trained from scratch with REINFORCE) makes **all non-combat decisions** — map pathing, card rewards, shops, campfires, events — while the simulator's built-in MCTS plays combat.
+
+## Headline result
+
+Same MCTS combat, same 50 held-out seeds, A0 Ironclad — only the non-combat "brain" differs:
+
+| non-combat decisions | combat | avg floor | win rate |
+|---|---|---|---|
+| stock bot heuristics (map = random) | MCTS @50000 | 31.2 | 6% |
+| **this model** | MCTS @50000 | **42.5** | **14%** |
+
+The learned non-combat layer is worth **~11 floors** over the stock bot: its biggest weakness was never combat, it was walking the map at random.
+
+## Files
+
+| file | what |
+|---|---|
+| `armG_model_G128x128_15k.pt` | **the non-combat policy** behind the headline number (15k games) |
+| `armG_model_G128x128.pt` | earlier 8k-game checkpoint |
+| `armS_card_vocab.json` | card vocabulary (required to encode candidates) |
+| `armB_model_B256x256.pt` | combat behavior-cloning model — **negative result** (floor ~12 vs teacher 23) |
+| `armB_model_VAL256x256.pt` | combat value net — negative result (1-ply lookahead: floor ~8) |
+| `armB_model_ATTN_d64L2.pt` | combat attention model — negative result (floor ~14) |
+
+The combat models are published on purpose: six different attempts to distill MCTS combat into a feed-forward network all failed the same way (imitation caps at 0.44 train accuracy — the MCTS teacher effectively sees the future draw order; a one-frame policy can't). *Judgment*-type decisions compress into small networks easily; *planning*-type decisions resist.
+
+## Usage
+
+Input is `obs(412) ⊕ candidate-descriptor(368)` → scalar score per candidate; pick the argmax. You need the patched simulator and the encoding code — **full code, sim patch, training scripts and eval protocol: [github.com/valiant-wjl/sts-rl-agent](https://github.com/valiant-wjl/sts-rl-agent)**.
+
+```python
+import torch
+from agent.armG_train import Scorer, build_choices, obs_vec   # from the GitHub repo
+net = Scorer((128, 128))
+net.load_state_dict(torch.load("armG_model_G128x128_15k.pt", weights_only=True))
+```
+
+## Limitations
+
+- A0 (lowest difficulty), Ironclad only (the simulator only fully implements Ironclad).
+- Combat is still search (MCTS), not learned.
+- Single-run numbers on 50 fixed seeds, no confidence intervals.
+
+Slay the Spire is a trademark of Mega Crit Games; this is an unaffiliated research project on a clean-room simulator (MIT).