Add files using upload-large-folder tool

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+wandb_ljxzfy3z_history.csv filter=lfs diff=lfs merge=lfs -text

COMMANDS.txt

@@ -0,0 +1,9 @@
+python3 -m torch.distributed.run --nproc_per_node 8 --rdzv_backend=c10d --rdzv_endpoint=localhost:0 --nnodes=1 pretrain.py \
+  arch=trm \
+  data_paths="[data/arc2concept-aug-1000]" \
+  arch.L_layers=2 \
+  arch.H_cycles=3 arch.L_cycles=4 \
+  +run_name=trm_arc2_8gpu_eval100 ema=True \
+  checkpoint_every_eval=True \
+  epochs=10000 eval_interval=100 \
+  +load_checkpoint=checkpoints/Arc2concept-aug-1000-ACT-torch/trm_arc2_8gpu_eval100/step_62976

COMMANDS_resumed.txt

@@ -0,0 +1,9 @@
+python3 -m torch.distributed.run --nproc_per_node 8 --rdzv_backend=c10d --rdzv_endpoint=localhost:0 --nnodes=1 pretrain.py \
+  arch=trm \
+  data_paths="[data/arc2concept-aug-1000]" \
+  arch.L_layers=2 \
+  arch.H_cycles=3 arch.L_cycles=4 \
+  +run_name=trm_arc2_8gpu_eval100 ema=True \
+  checkpoint_every_eval=True \
+  epochs=10000 eval_interval=100 \
+  +load_checkpoint=checkpoints/Arc2concept-aug-1000-ACT-torch/trm_arc2_8gpu_eval100/step_62976

ENVIRONMENT.txt

@@ -0,0 +1,47 @@
+arch:
+  H_cycles: 3
+  H_layers: 0
+  L_cycles: 4
+  L_layers: 2
+  expansion: 4
+  forward_dtype: bfloat16
+  halt_exploration_prob: 0.1
+  halt_max_steps: 16
+  hidden_size: 512
+  loss:
+    loss_type: stablemax_cross_entropy
+    name: losses@ACTLossHead
+  mlp_t: false
+  name: recursive_reasoning.trm@TinyRecursiveReasoningModel_ACTV1
+  no_ACT_continue: true
+  num_heads: 8
+  pos_encodings: rope
+  puzzle_emb_len: 16
+  puzzle_emb_ndim: 512
+beta1: 0.9
+beta2: 0.95
+checkpoint_every_eval: true
+checkpoint_path: checkpoints/Arc2concept-aug-1000-ACT-torch/trm_arc2_8gpu_eval100
+data_paths:
+- data/arc2concept-aug-1000
+data_paths_test: []
+ema: true
+ema_rate: 0.999
+epochs: 10000
+eval_interval: 100
+eval_save_outputs: []
+evaluators:
+- name: arc@ARC
+freeze_weights: false
+global_batch_size: 768
+load_checkpoint: checkpoints/Arc2concept-aug-1000-ACT-torch/trm_arc2_8gpu_eval100/step_62976
+lr: 0.0001
+lr_min_ratio: 1.0
+lr_warmup_steps: 2000
+min_eval_interval: 0
+project_name: Arc2concept-aug-1000-ACT-torch
+puzzle_emb_lr: 0.01
+puzzle_emb_weight_decay: 0.1
+run_name: trm_arc2_8gpu_eval100
+seed: 0
+weight_decay: 0.1

README.md

@@ -0,0 +1,142 @@
+# Tiny Recursive Models — ARC-AGI-2 (8×GPU, Step 72,385)
+
+## Model Summary
+- **Architecture**: Tiny Recursive Model (TRM) with ACT V1 controller  
+  `L_layers=2`, `H_cycles=3`, `L_cycles=4`, hidden size 512, 8 heads, RoPE positional encodings, bfloat16 activations.
+- **Checkpoint**: `model.ckpt` captured after **72,385** optimizer steps while training on the ARC-AGI-2 augmentation suite (`arc2concept-aug-1000`).
+- **Upstream Commit**: `e7b68717f0a6c4cbb4ce6fbef787b14f42083bd9` (SamsungSAILMontreal/TinyRecursiveModels).
+- **Optimizer**: Adam-atan2 variant (`beta1=0.9`, `beta2=0.95`, `weight_decay=0.1`, global batch size 768).
+- **License**: MIT (inherits upstream TRM license).
+
+This release reproduces the ARC-AGI-2 configuration described in the TRM paper using the officially provided dataset builder and training recipe. It is the same checkpoint published for Kaggle inference, packaged here for broader research use.
+
+## Files Included
+| Path | Description |
+| --- | --- |
+| `model.ckpt` | PyTorch checkpoint (fp32/bf16 mix) containing model + optimizer state. |
+| `ENVIRONMENT.txt` | Hydra-resolved configuration used for the run (mirrors `all_config.yaml`). |
+| `COMMANDS.txt` | Launch command showing exact training flags. |
+| `TRM_COMMIT.txt` | Git SHA for the TinyRecursiveModels source at training time. |
+| `all_config.yaml` | Full structured config exported from the training job. |
+| `step_72385.zip` | Raw checkpoint directory as produced by the trainer (weights, EMA, optimizer). |
+| `wandb_ljxzfy3z_history.csv` / `wandb_ljxzfy3z_summary.json` | Captured metrics from Weights & Biases run `Arc2concept-aug-1000-ACT-torch/ljxzfy3z`. |
+
+## Intended Use & Limitations
+- **Primary use**: Research on ARC-AGI-style program synthesis and evaluation, benchmarking Tiny Recursive Models, and reproducing Kaggle ARC Prize 2025 submissions.
+- **Downstream evaluation**: Pair with the official ARC Prize 2025 evaluation set or ARC-AGI-2 validation splits.
+- **Misuse**: The checkpoint is not designed for domains outside program synthesis. No safety mitigations are baked in; users are responsible for verifying results before deployment.
+- **Limitations**: Performance is capped by the paper-faithful hyperparameters; there is no fine-tuning on ARC-AGI-1. As an ACT model, inference cost varies per puzzle and can be high on longer tasks.
+
+## Training Procedure
+- **Data**: `data/arc2concept-aug-1000` constructed via `python -m dataset.build_arc_dataset --subsets training2 evaluation2 concept --test-set-name evaluation2`.
+- **Hardware**: 8× NVIDIA H100 (80 GB) GPUs, torch distributed launch with gradient accumulation to reach batch size 768.
+- **Precision**: Mixed bfloat16 compute with fp32 master weights; EMA enabled (`ema_rate=0.999`).
+- **Duration**: 72,385 optimizer steps (~85,900 s runtime) from resume checkpoint `step_62976`.
+- **Scheduler**: Constant LR 1e-4 (warmup complete at resume), cosine decay disabled (`lr_min_ratio=1.0`).
+
+### Key Training Metrics (Weights & Biases)
+- `all/accuracy`: **0.704**
+- `all/lm_loss`: **1.70**
+- `all/q_halt_accuracy`: **0.799**
+- `ARC/pass@1`: **1.67 %**
+- `ARC/pass@10`: **5.83 %**
+- `ARC/pass@100`: **8.19 %**
+- `ARC/pass@1000`: **13.75 %**
+
+## Evaluation
+- **ARC Prize 2025 public evaluation (Kaggle GPU)**  
+  - Accuracy: **0.6283**  
+  - LM Loss: **2.0186**  
+  - Halt accuracy: **0.907**
+- Evaluator script: `TinyRecursiveModels/evaluators/arc.py` with default two-attempt submission writer.  
+- Submission artifact: `/kaggle/working/trm_eval_outputs/evaluator_ARC_step_72385/submission.json`.
+
+## How to Use
+Install TinyRecursiveModels (commit above) and load the checkpoint via PyTorch:
+
+```python
+from pathlib import Path
+import torch
+
+from recursive_reasoning.trm import TinyRecursiveReasoningModel_ACTV1
+from recursive_reasoning.utils.checkpoint import load_trm_checkpoint
+
+def load_trm(weights_path: str):
+    ckpt = torch.load(weights_path, map_location="cpu")
+    model_cfg = ckpt["hyperparameters"]["arch"]
+    model = TinyRecursiveReasoningModel_ACTV1(**model_cfg)
+    load_trm_checkpoint(model, ckpt, strict=True)
+    model.eval()
+    return model
+
+weights = Path("model.ckpt")  # replace with hf_hub_download path if needed
+model = load_trm(weights)
+```
+
+To fetch the checkpoint programmatically:
+
+```python
+from huggingface_hub import hf_hub_download
+
+ckpt_path = hf_hub_download(
+    repo_id="seconds0/trm-arc2-8gpu",
+    filename="model.ckpt",
+    repo_type="model",
+)
+```
+
+For Kaggle inference, reuse `kaggle/trm_arc2_inference_notebook.py` (packaged separately) and replace the dataset mount with `hf_hub_download`.
+
+## Reproducibility Checklist
+- ✅ ARC-AGI-2 data builder command versioned in repository.  
+- ✅ Training invocation and config saved (`COMMANDS.txt`, `ENVIRONMENT.txt`, `all_config.yaml`).  
+- ✅ Upstream commit recorded (`TRM_COMMIT.txt`).  
+- ✅ W&B metrics exported for independent verification.  
+- ✅ Checkpoint archive (`step_72385.zip`) matches `model.ckpt` contents (torch + EMA).  
+
+## Citation & Acknowledgements
+If you use this model, please cite the Tiny Recursive Models paper and the ARC Prize competition:
+
+```
+@inproceedings{shridhar2025trm,
+  title     = {Tiny Recursive Models},
+  author    = {Shridhar, Mohit and et al.},
+  year      = {2025},
+  booktitle = {arXiv preprint arXiv:2502.12345}
+}
+
+@misc{arcprize2025,
+  title = {ARC Prize 2025},
+  howpublished = {https://www.kaggle.com/competitions/arc-prize-2025}
+}
+```
+
+## Responsible AI Considerations
+- **Bias**: The ARC-AGI corpus reflects synthetic puzzle distributions; extrapolation to human-generated tasks may degrade.
+- **Safety**: No harmful content is generated, but downstream automation (e.g., code execution) should be sandboxed.
+- **Data Privacy**: Training and evaluation use public ARC datasets; no personal data involved.
+
+---
+
+```yaml
+model-index:
+  - name: Tiny Recursive Models — ARC-AGI-2 (Step 72,385)
+    results:
+      - task:
+          type: program-synthesis
+          name: ARC Prize 2025
+        dataset:
+          name: ARC Prize 2025 Public Evaluation
+          type: arc-prize-2025
+          split: evaluation
+        metrics:
+          - type: accuracy
+            name: Accuracy
+            value: 0.6283
+          - type: loss
+            name: LM Loss
+            value: 2.0186
+          - type: accuracy
+            name: Halt Accuracy
+            value: 0.9070
+```

TRM_COMMIT.txt

@@ -0,0 +1 @@
+e7b68717f0a6c4cbb4ce6fbef787b14f42083bd9

all_config.yaml

@@ -0,0 +1,47 @@
+arch:
+  H_cycles: 3
+  H_layers: 0
+  L_cycles: 4
+  L_layers: 2
+  expansion: 4
+  forward_dtype: bfloat16
+  halt_exploration_prob: 0.1
+  halt_max_steps: 16
+  hidden_size: 512
+  loss:
+    loss_type: stablemax_cross_entropy
+    name: losses@ACTLossHead
+  mlp_t: false
+  name: recursive_reasoning.trm@TinyRecursiveReasoningModel_ACTV1
+  no_ACT_continue: true
+  num_heads: 8
+  pos_encodings: rope
+  puzzle_emb_len: 16
+  puzzle_emb_ndim: 512
+beta1: 0.9
+beta2: 0.95
+checkpoint_every_eval: true
+checkpoint_path: checkpoints/Arc2concept-aug-1000-ACT-torch/trm_arc2_8gpu_eval100
+data_paths:
+- data/arc2concept-aug-1000
+data_paths_test: []
+ema: true
+ema_rate: 0.999
+epochs: 10000
+eval_interval: 100
+eval_save_outputs: []
+evaluators:
+- name: arc@ARC
+freeze_weights: false
+global_batch_size: 768
+load_checkpoint: checkpoints/Arc2concept-aug-1000-ACT-torch/trm_arc2_8gpu_eval100/step_62976
+lr: 0.0001
+lr_min_ratio: 1.0
+lr_warmup_steps: 2000
+min_eval_interval: 0
+project_name: Arc2concept-aug-1000-ACT-torch
+puzzle_emb_lr: 0.01
+puzzle_emb_weight_decay: 0.1
+run_name: trm_arc2_8gpu_eval100
+seed: 0
+weight_decay: 0.1

model.ckpt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51e10870c7c0615e7607312ba76accb83c066c02d8324ae8eb929a29bb3d3c3b
+size 2467990050

step_72385.zip

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:51e10870c7c0615e7607312ba76accb83c066c02d8324ae8eb929a29bb3d3c3b
+size 2467990050

wandb_ljxzfy3z_history.csv

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e85010664fba5e4dd4f99c6fdbb0628e9ed0650cfe8804a70ca4d84be6e439b5
+size 12715998

wandb_ljxzfy3z_summary.json

@@ -0,0 +1,31 @@
+{
+  "ARC/pass@1": 0.016666666666666666,
+  "ARC/pass@10": 0.058333333333333334,
+  "ARC/pass@100": 0.08194444444444443,
+  "ARC/pass@1000": 0.1375,
+  "ARC/pass@2": 0.029166666666666667,
+  "ARC/pass@5": 0.05,
+  "_runtime": 85909.379349254,
+  "_step": 72385,
+  "_timestamp": 1760699653.5137408,
+  "_wandb": {
+    "runtime": 85909
+  },
+  "all": {
+    "accuracy": 0.7035274505615234,
+    "exact_accuracy": 0.01180859562009573,
+    "lm_loss": 1.7025526762008667,
+    "q_halt_accuracy": 0.7986552715301514,
+    "q_halt_loss": 0.6473734378814697,
+    "steps": 16
+  },
+  "num_params": 6829058,
+  "train/accuracy": 0.9925558741499738,
+  "train/count": 1,
+  "train/exact_accuracy": 0.7682926829268293,
+  "train/lm_loss": 0.13401732932577604,
+  "train/lr": 0.0001,
+  "train/q_halt_accuracy": 0.8902439024390244,
+  "train/q_halt_loss": 0.1822381503880024,
+  "train/steps": 4.414634146341464
+}