File size: 1,708 Bytes
138a6a8 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | ---
tags:
- arc-agi
- abstract-reasoning
- rule-conditioned-transformer
- discrete-reasoning
license: mit
---
# ARC-IT: Rule-Conditioned Transformer for ARC-AGI
A novel architecture that solves abstract reasoning tasks (ARC-AGI) by explicitly
extracting transformation rules from demonstration pairs and applying them to new inputs:
- **GridTokenizer** -- Embeds discrete ARC grids (0-11) into continuous patch tokens
- **RuleEncoder** -- Extracts transformation rules from demo input/output pairs via cross-attention
- **RuleApplier** -- Applies the learned rules to a test input via cross-attention
- **SpatialDecoder** -- Converts output tokens to 64x64 grid logits
## Architecture
```
Demo Pairs -> GridTokenizer -> RuleEncoder (cross-attention + aggregation) -> Rule Tokens
Test Input -> GridTokenizer -> RuleApplier (cross-attention to rules) -> SpatialDecoder -> Predicted Grid
```
## Training
- **2-stage training**: Full Training -> Hard Focus (AGI-2 oversampling)
- **Test-Time Training (TTT)**: Per-task fine-tuning on demonstration examples
## Model Details
- **Training step**: 18000
- **Best validation accuracy**: 0.733029360572497
- **Hidden size**: 384
- **Rule Encoder**: 2 pair layers, 2 agg layers, 64 rule tokens
- **Rule Applier**: 4 layers, 8 heads
- **Canvas size**: 64
## Usage
```python
import torch
from arc_it.models.arc_it_model import ARCITModel
model = ARCITModel.from_config(config)
ckpt = torch.load("model.pt", map_location="cpu", weights_only=False)
model.load_state_dict(ckpt["model_state_dict"])
```
## Links
- **Repository**: [github.com/REDDITARUN/arc_it](https://github.com/REDDITARUN/arc_it)
- **ARC-AGI**: [arcprize.org](https://arcprize.org)
|