---
tags:
  - arc-agi
  - abstract-reasoning
  - rule-conditioned-transformer
  - discrete-reasoning
license: mit
---

# ARC-IT: Rule-Conditioned Transformer for ARC-AGI

A novel architecture that solves abstract reasoning tasks (ARC-AGI) by explicitly
extracting transformation rules from demonstration pairs and applying them to new inputs:

- **GridTokenizer** -- Embeds discrete ARC grids (0-11) into continuous patch tokens
- **RuleEncoder** -- Extracts transformation rules from demo input/output pairs via cross-attention
- **RuleApplier** -- Applies the learned rules to a test input via cross-attention
- **SpatialDecoder** -- Converts output tokens to 64x64 grid logits

## Architecture

```
Demo Pairs -> GridTokenizer -> RuleEncoder (cross-attention + aggregation) -> Rule Tokens
Test Input  -> GridTokenizer -> RuleApplier (cross-attention to rules) -> SpatialDecoder -> Predicted Grid
```

## Training

- **2-stage training**: Full Training -> Hard Focus (AGI-2 oversampling)
- **Test-Time Training (TTT)**: Per-task fine-tuning on demonstration examples

## Model Details

- **Training step**: 18000
- **Best validation accuracy**: 0.733029360572497
- **Hidden size**: 384
- **Rule Encoder**: 2 pair layers, 2 agg layers, 64 rule tokens
- **Rule Applier**: 4 layers, 8 heads
- **Canvas size**: 64

## Usage

```python
import torch
from arc_it.models.arc_it_model import ARCITModel

model = ARCITModel.from_config(config)
ckpt = torch.load("model.pt", map_location="cpu", weights_only=False)
model.load_state_dict(ckpt["model_state_dict"])
```

## Links

- **Repository**: [github.com/REDDITARUN/arc_it](https://github.com/REDDITARUN/arc_it)
- **ARC-AGI**: [arcprize.org](https://arcprize.org)