Create README.md

README.md

@@ -0,0 +1,148 @@
+---
+tags:
+  - transgrid
+  - systematic-generalization
+datasets:
+  - BlueWhaleLab/TranSGrid
+---
+
+# TranSGrid Models
+
+This repository provides the trained Transformer checkpoints for **TranSGrid**, introduced in [What Do Current Systematic Generalization Tasks Miss? A Reasoning-Centered Analysis](https://arxiv.org/abs/2609.19212).
+
+TranSGrid is a grid-transformation testbed for studying systematic generalization through **deductive, inductive, and abductive reasoning**. Given an initial board and a target board, a model needs to generate a sequence of actions that transforms the initial board into the target. The action space includes ten operation types on rows, columns, and local 2 × 2 blocks.
+
+The released models operate on **6 × 6 boards containing digits from 0 to 9**. Predictions are evaluated by executing the generated actions: a prediction is correct if it reaches the target board, even if it differs from the reference sequence.
+
+**Links:** [Paper](https://arxiv.org/abs/2609.19212) · [Code](https://github.com/BlueWhaleLab/TranSGrid) · [Dataset](https://huggingface.co/datasets/BlueWhaleLab/TranSGrid) · [Interactive Demo](https://huggingface.co/spaces/BlueWhaleLab/TranSGrid)
+
+## Available Models
+
+Checkpoints are organized into four model configurations:
+
+| Directory | Configuration |
+| --- | --- |
+| `flat_board/` | FLAT + BOARD |
+| `flat_pair/` | FLAT + PAIR |
+| `grid_board/` | GRID + BOARD |
+| `grid_pair/` | GRID + PAIR |
+
+Each configuration contains seven encoder–decoder Transformer checkpoints, named `level1.pt` through `level7.pt`. The GRID+PAIR models range from **0.96M** parameters at Level 1 to **88.43M** parameters at Level 7.
+The examples below use `grid_pair/level7.pt`. To run another model, download its checkpoint and update the `--ckpt` argument.
+
+## Installation
+
+The checkpoints are PyTorch `.pt` files. Use them with the model implementation and evaluation scripts in the [GitHub repository](https://github.com/BlueWhaleLab/TranSGrid).
+
+Python **3.10 or newer** is required. Run all subsequent commands from the code repository's root directory.
+
+```bash
+git clone https://github.com/BlueWhaleLab/TranSGrid.git
+cd TranSGrid
+
+conda create -p ./.conda python=3.12
+conda activate ./.conda
+
+pip install -r requirements.txt
+pip install huggingface_hub
+```
+
+## Download a Checkpoint
+
+Download the GRID+PAIR Level 7 model:
+
+```bash
+hf download BlueWhaleLab/TranSGrid \
+  grid_pair/level7.pt \
+  --local-dir checkpoints
+```
+
+The checkpoint will be saved to `checkpoints/grid_pair/level7.pt`.
+
+For example, to download the FLAT+BOARD Level 3 model:
+
+```bash
+hf download BlueWhaleLab/TranSGrid \
+  flat_board/level3.pt \
+  --local-dir checkpoints
+```
+
+## Run Evaluation
+
+### Greedy Decoding
+
+Evaluate the model on the TranSGrid benchmark:
+
+```bash
+python eval.py \
+  --ckpt checkpoints/grid_pair/level7.pt \
+  --data data/transgrid.jsonl \
+  --beams 1 \
+  --batch-size 128 \
+  --save results/TranSGrid/level7_grid_pair-greedy-bs128.jsonl \
+  --group-by deductive_score inductive_score abductive_level stop_reason
+```
+
+### Beam Search (Top@8)
+
+Generate eight candidates per instance:
+
+```bash
+python eval.py \
+  --ckpt checkpoints/grid_pair/level7.pt \
+  --data data/transgrid.jsonl \
+  --beams 8 \
+  --batch-size 128 \
+  --save results/TranSGrid/level7_grid_pair-beam8-bs128.jsonl \
+  --group-by deductive_score inductive_score abductive_level stop_reason
+```
+
+**Top@8** counts an instance as solved if any of the eight candidates reaches the target board. Each saved JSONL row contains one selected prediction, its solve status, and the rank of the first successful candidate; it does not contain all eight candidates.
+
+Reduce `--batch-size` if GPU memory is limited.
+
+### Held-out Test Set
+
+Download the held-out Test split:
+
+```bash
+hf download BlueWhaleLab/TranSGrid \
+  test.jsonl \
+  --repo-type dataset \
+  --local-dir data
+```
+
+Then evaluate it:
+
+```bash
+python eval.py \
+  --ckpt checkpoints/grid_pair/level7.pt \
+  --data data/test.jsonl \
+  --beams 8 \
+  --batch-size 128 \
+  --save results/Test/level7_grid_pair-beam8-bs128.jsonl
+```
+
+### View Saved Results
+
+Print aggregate and grouped results without running decoding again:
+
+```bash
+python report.py results/TranSGrid \
+  --axes deductive_score inductive_score abductive_level stop_reason
+```
+
+For dataset generation, model training, and evaluation on the **Decoupled** and **SCAN** variants, see the [GitHub README](https://github.com/BlueWhaleLab/TranSGrid#readme).
+
+## Citation
+
+If you use these models or TranSGrid in your research, please cite:
+
+```bibtex
+@article{qi2026current,
+  title={What Do Current Systematic Generalization Tasks Miss? A Reasoning-Centered Analysis},
+  author={Qi, Chengwen and Ye, Deheng and Bian, Yatao},
+  journal={arXiv preprint arXiv:2609.19212},
+  year={2026}
+}
+```