FPRM — Maze-Hard 30×30 (single-z, conv1d)

Fixed-Point Tiny Recursive Model (FPRM / FPTRM single-z) trained on Maze-Hard 30×30. Under test-time scaling it reaches 87.0% test exact-match (full-path sequence accuracy), the best of our maze runs.

Original run name: fptrm_singlez_maze_isolrev_repro_warmup2000 (step_78120 = final).

Results

metric	value
test exact-match (sequence acc)	87.0% (best of 6 eval-restart seeds; mean 86.6%)
token accuracy	99.57%
eval budget	`max_iter=35000`, `stepsize_decay=0.996`, `patience=10`

Per-seed scaling-eval results are in eval_results/ (seeds 0–5: 86.6 / 86.5 / 86.4 / 86.6 / 87.0 / 86.6 %). The conv1d variant here and a no-conv variant both peak at 87.0%; a conv2d variant reaches 84.5% — i.e. the conv branch is not the deciding factor on maze.

⚠️ Data note (read before reproducing)

This model was trained on the non-augmented Maze-Hard set (1000 mazes, single orientation). The training config records data_paths: [data/maze-30x30-hard-1k], but in our pipeline that path later became the 8× dihedral-augmented build, which collapses maze training to ~5% (augmentation breaks this task for a non-equivariant model). To reproduce, train on the non-augmented 1000-maze build. The test split is identical (single orientation) either way.

Architecture

	value
model	FPTRM single-z (`fp_trm_singlez`)
hidden size / heads / expansion	512 / 8 / 4
H-cycles / L-cycles	0 / 0
H-layers / L-layers	0 / 2
n_backwards_L	6
positional enc / puzzle-emb len	RoPE / 16
halting	fixed-point
max_iter (train)	24 (deterministic)
max_iter (eval)	1000 default; 35000 for the scaling result
stepsize-decay / patience (eval)	0.996 / 10 (train-time default 0.9 / 5)
fp_thresh	0.1
norm type / placement	pre-norm / output
residual scaling	input-independent (α₁, α₂ init = 0.75, 0.25)
conv branch	conv1d (kernel 4)
init_std	1.0

Optimizer

	value
optimizer	AdamW (β = 0.9, 0.95)
learning rate	1e-4 (constant after 2000-step warm-up)
weight decay	1.0
puzzle-emb LR	1e-2 (decoupled)
puzzle-emb WD	1.0
batch size	768
epochs	60000
EMA	enabled (rate 0.999)
seed	0

Note: the optimizer is AdamW (adam_atan2 was not used for this checkpoint).

Files

step_<N> — EMA-averaged eval checkpoints (53 MB each), N = 6510 … 78120 (final). Because ema=True, step_<N> is the EMA copy — these are the weights to load for inference/eval.
step_<N>_train_state.pt — full training state (live weights + optimizer + EMA), for resuming training.
all_config.yaml — the exact training configuration.
fp_trm_singlez.py, losses.py — model and loss source for this checkpoint.
reproduce/ — training SLURM script (submit_fptrm_maze_87pct_repro_8gpu.sh) and the eval script (singlez_eval_sweep.py).
eval_results/ — the per-seed scaling-eval JSONs that produced the 87%.

Reproduce the 87%

Train with reproduce/submit_fptrm_maze_87pct_repro_8gpu.sh (FPTRM single-z, conv1d/k4, pre-norm/output, input-independent residual scaling, decoupled puzzle-emb LR=1e-2, 60000 epochs, non-augmented maze). Produces step_78120.

Eval at the 35k budget on the resulting checkpoint:

python singlez_eval_sweep.py --ckpt-dir <CKPT_DIR> \
    --data-dir <maze-hard-test> \
    --stepsize-decay 0.996 --decay-patience 10 --max-iter 35000 --seed 4

87.0% is the best of restart seeds 0–5 (mean ≈ 86.6%).