LAW (Loopholing All the Way) β Sudoku-hard checkpoints (CS-433)
Three training seeds of LAW, a continuous diffusion language model whose denoising loop feeds back the expected embedding of each step's prediction, trained on hard Sudoku.
| folder | training seed | accuracy (2,000 validation puzzles, 180 NFE, T = 1, 8 sampling seeds) |
|---|---|---|
seed1/ |
1 | 95.7 % |
seed2/ |
2 | 95.9 % |
seed3/ |
3 | 94.4 % |
Mean 95.3 % (the paper reports 96.5 % for this configuration).
Recipe
The paper's best hard-Sudoku configuration: model tiny-loop (8 DiT blocks, width 512,
sequence length 180), algo=law, cross-entropy loss, pass counts M = 2 with
p = (0.1, 0.9), last-pass objective, stop-gradient between passes, time advanced between
passes (dt = 1/180), carry-temperature regularization Ο = 2, batch 256, 15,385 steps.
Each folder holds the final PyTorch Lightning checkpoint (81-15385.ckpt, with optimizer
state and EMA weights) and the Hydra configuration it was trained with
(config.yaml, overrides.yaml).
Loading
import algo, dataloader
from hydra import compose, initialize_config_dir
with initialize_config_dir(version_base=None, config_dir='<repo>/configs'):
config = compose(config_name='config', overrides=[...]) # see seedN/overrides.yaml
tokenizer = dataloader.get_tokenizer(config)
model = algo.LAW.load_from_checkpoint('seed1/81-15385.ckpt', tokenizer=tokenizer, config=config)
model._eval_mode() # EMA weights