arc2/README.md

# FPRM — ARC-2-concept (single-z, conv1d)

Fixed-Point Tiny Recursive Model (FPRM / FPTRM single-z) trained on **ARC-2-concept**
(`arc2concept-aug-1000`, the 1000-task augmented build). Evaluated with two attempts it
reaches **6.2% pass@2** — ARC-2-concept is substantially harder than ARC-1-concept under
the same recipe.

Original run name: `fptrm_singlez_arc2_lr1e-3_wd0.01_det_a0.75_0.25_iter8` (step_723914 = final;
resumed run).

## Results

| metric | value |
|---|---|
| **test pass@2** | **6.2%** |
| eval budget | `max_iter_eval=1000` (stepsize_decay 0.9, patience 5) |
| checkpoint | `step_723914` (final, EMA) |

## Architecture

| | value |
|---|---|
| model | FPTRM single-z (`fp_trm_singlez`) |
| hidden size / heads / expansion | 512 / 8 / 4 |
| H-cycles / L-cycles | 0 / 0 |
| H-layers / L-layers | 0 / 2 |
| n_backwards_L | 6 |
| positional enc / puzzle-emb len | RoPE / 16 |
| halting | fixed-point + Q-head (ACT) |
| max_iter (train) | 8 (deterministic) |
| max_iter (eval) | 1000 |
| stepsize-decay / patience | 0.9 / 5 |
| fp_thresh | 0.1 |
| norm type / placement | pre-norm / none |
| residual scaling | input-independent (α₁, α₂ init = 0.75, 0.25) |
| conv branch | conv1d (kernel 4) |
| loss | stablemax cross-entropy, deep supervision, q_loss_coeff 0.5 |

## Optimizer

| | value |
|---|---|
| optimizer | adam_atan2 (β = 0.9, 0.95) |
| learning rate | 1e-3 (constant, no warm-up) |
| weight decay | 0.01 |
| **puzzle-emb LR** | **1e-2 (decoupled)** |
| puzzle-emb WD | 1.0 |
| batch size | 768 |
| epochs | 100000 |
| EMA | enabled (rate 0.999) |
| seed | 0 |

## Files

- `step_<N>` — EMA-averaged eval checkpoints (~1.7 GB each), N = 72391 … **723914** (final).
  Because `ema=True`, `step_<N>` is the EMA copy — **these are the weights to load for
  inference/eval**.
- `all_config.yaml` — the exact training configuration.
- `fp_trm_singlez.py`, `losses.py` — model and loss source for this checkpoint.
- `evaluator_ARC_step_0/submission.json` — ARC evaluation submission (predictions) for this run.

## Reproduce the 6.2%

1. **Train** with `all_config.yaml` (FPTRM single-z, conv1d/k4, pre-norm / placement none,
   input-independent residual scaling, `adam_atan2`, lr 1e-3, decoupled puzzle-emb LR 1e-2,
   100000 epochs) on the **`arc2concept-aug-1000`** build. Produces `step_723914`.
2. **Eval at `max_iter=1000`** on the final EMA checkpoint and report **pass@2** (two attempts).