README.md

---
library_name: pytorch
license: mit
pipeline_tag: other
tags:
  - arc-prize-2025
  - program-synthesis
  - tiny-recursive-models
  - recursive-reasoning
  - resume-training
  - act
  - reproducibility
datasets:
  - arc-prize-2025
model-index:
  - name: Tiny Recursive Models — ARC-AGI-2 (Resume Step 119432)
    results:
      - task:
          type: program-synthesis
          name: ARC Prize 2025 (legacy evaluation mapping)
        dataset:
          name: ARC Prize 2025 Public Evaluation
          type: arc-prize-2025
          split: evaluation
        metrics:
          - type: accuracy
            name: ARC Task Solve Rate (pass@1)
            value: 0.0083
          - type: accuracy
            name: ARC Task Solve Rate (pass@2)
            value: 0.0083
          - type: accuracy
            name: ARC Task Solve Rate (pass@10)
            value: 0.0083
          - type: accuracy
            name: ARC Task Solve Rate (pass@100)
            value: 0.0083
---

# Tiny Recursive Models — ARC-AGI-2 (8× H200 Resume, Step 119 432)

Overview
- 8× H200 resume snapshot at global step 119 432 from run `trm_arc2_8gpu_resume_step115815_plus100k_v2` (TinyRecursiveModels commit `e7b68717`).
- Consolidated `model.ckpt` with accompanying configuration and provenance files. Integrity hash: `2bc8bb3a5a85cd73e169a6fd285f9138427db894bd157edc20e92a58ed8ee33e`.
- Current evaluation on the ARC public evaluation split is approximately 0.83% pass@1, attributable to duplicate candidate generation in our evaluation pipeline after the resume. This release is intended for reproducibility and analysis rather than leaderboard submissions.

About “119 434” vs “119 432”
- Internal tracking referenced “step 119 434”; the persistent shard is `step_119432`. W&B logs confirm the resume guard initialized at step 115 815 and advanced into the 119k block. No distinct 119 434 shard remains.

Contents
- `model.ckpt` — Consolidated PyTorch checkpoint (weights, optimizer, EMA) reflecting `step_119432/*`.
- `COMMANDS.txt`, `COMMANDS_resumed.txt` — Exact `torchrun` invocations (8× H200) and resume parameters.
- `ENVIRONMENT.txt`, `all_config.yaml` — Hydra-resolved configurations captured on the training pod.
- `MANIFEST.txt` — Packaging metadata (step, source path, timestamp, sha256).
- `TRM_COMMIT.txt` — Upstream TinyRecursiveModels commit (`e7b68717f0a6c4cbb4ce6fbef787b14f42083bd9`).
- `dataset-metadata.json` — Kaggle packaging manifest (legacy identifier mapping). W&B CSV/summary are included for convenience.

Kaggle Status
- This checkpoint is not accompanied by a validated Kaggle submission from our pipeline. In our evaluation path, inference produced duplicate candidates across attempts, yielding ≈0.83% pass@1 on the public evaluation split.
- A third‑party Kaggle implementation demonstrating TRM inference is available on Kaggle; interested users can locate it by searching the platform. That implementation is independent of this release.

Current Kaggle Limitations
- Unique‑candidate enforcement: A production overlay inadvertently bypassed the uniqueness filter, allowing duplicates even with `ARC_SAMPLING_COUNT > 1`.
- Sampler configuration propagation: Variables such as `ARC_SAMPLING_COUNT`, `ARC_SAMPLING_MODE=sample`, and temperature did not reliably reach the evaluator under our overlays.
- Sampling degeneracy: Under misconfiguration, multinomial sampling collapsed to identical outputs across attempts.
- Limited attempt‑level telemetry: Evaluators emitted aggregate metrics only; per‑attempt logits and strings were not retained due to GPU runtime constraints and Kaggle logging limits. This prevents visualization of candidate selection and diversity.
- Identifier mapping: The checkpoint was trained with the legacy identifier mapping, whereas some evaluators assume the sorted mapping. Without remapping or a compatibility layer, comparisons can be brittle.

Attempted Mitigations
- Relaunched controlled evaluator pods with explicit sampling parameters and verified resume‑guard logs; duplicates persisted.
- Instrumented CPU/GPU debug evaluators (`scripts/debug_eval_cpu.py --samples 8 --log-attempts`); candidate diversity remained near zero.
- Adjusted temperature and top‑k settings; no material improvement under the broken overlay path.
- Prepared Kaggle datasets/notebooks and executed end‑to‑end; duplicate attempts persisted and no leaderboard submission was made.

Evaluation Guidance
- Enforce a strict unique‑candidate guard prior to scoring.
- Validate that sampling environment variables propagate into the inference process.
- Capture per‑attempt outputs and, where possible, logits to diagnose diversity.
- Match the identifier mapping to the checkpoint (legacy vs sorted) and use the corresponding dataset builder.

Reproduction
```python
from huggingface_hub import hf_hub_download
import torch

ckpt_path = hf_hub_download("seconds-0/trm-arc2-8gpu", "model.ckpt")
state = torch.load(ckpt_path, map_location="cpu")
print(state["hyperparameters"]["arch"]["hidden_size"])  # 512
```

```
# CoreWeave resume (reference)
kubectl apply -f infra/kubernetes/trm-train-8gpu-resume.yaml
# Requires configmaps: trm-common-script, trm-pyshim-cm, trm-eval-overlay-cm
```

Resume Guard Signals
- `RESUME_CHECKPOINT_PATH=/workspace/TinyRecursiveModels/checkpoints/Arc2concept-aug-1000-ACT-torch/trm_arc2_8gpu_resume_plus100k/step_115815`
- `RESUME_EXPECTED_STEP=115815`
- Pod logs contain: `[resume] initializing train_state.step to 115815` before training proceeds.

Known Issues and Next Steps
- Duplicate candidate generation: restore uniqueness enforcement, validate environment propagation, and re‑verify multinomial sampling.
- Identifier mapping mismatch: this release remains “legacy”; sorted‑mapping evaluations require remapping or fine‑tuning.
- Observability: add candidate‑level telemetry and visualizations to support reliable evaluation claims.

Ethics, License, and Intended Use
- MIT license. Intended for research and educational use. Users should independently validate evaluation protocols and candidate diversity prior to reporting results.

Acknowledgements
- Built on the Tiny Recursive Models codebase and evaluated against ARC Prize 2025 materials. A community‑maintained Kaggle implementation demonstrates a functioning TRM inference pipeline and can be located via search on Kaggle.