LAW (Loopholing All the Way) β€” Sudoku-hard checkpoints (CS-433)

Three training seeds of LAW, a continuous diffusion language model whose denoising loop feeds back the expected embedding of each step's prediction, trained on hard Sudoku.

folder training seed accuracy (2,000 validation puzzles, 180 NFE, T = 1, 8 sampling seeds)
seed1/ 1 95.7 %
seed2/ 2 95.9 %
seed3/ 3 94.4 %

Mean 95.3 % (the paper reports 96.5 % for this configuration).

Recipe

The paper's best hard-Sudoku configuration: model tiny-loop (8 DiT blocks, width 512, sequence length 180), algo=law, cross-entropy loss, pass counts M = 2 with p = (0.1, 0.9), last-pass objective, stop-gradient between passes, time advanced between passes (dt = 1/180), carry-temperature regularization ρ = 2, batch 256, 15,385 steps. Each folder holds the final PyTorch Lightning checkpoint (81-15385.ckpt, with optimizer state and EMA weights) and the Hydra configuration it was trained with (config.yaml, overrides.yaml).

Loading

import algo, dataloader
from hydra import compose, initialize_config_dir
with initialize_config_dir(version_base=None, config_dir='<repo>/configs'):
    config = compose(config_name='config', overrides=[...])  # see seedN/overrides.yaml
tokenizer = dataloader.get_tokenizer(config)
model = algo.LAW.load_from_checkpoint('seed1/81-15385.ckpt', tokenizer=tokenizer, config=config)
model._eval_mode()  # EMA weights
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support