BCT: Bayes-Consistent Tempering
for Flow Language Models

Daisuke Oba, Hiroki Furuta, Naoaki Okazaki

Checkpoints trained for the paper BCT: Bayes-Consistent Tempering for Flow Language Models.

All models build on S-FLM (Language Modeling with Hyperspherical Flows). Checkpoints that we use without modification (the released S-FLM, MDLM and Duo TinyGSM models) are not included here; download them from jdeschena/s-flm (see below).

Files

File	Task	Backbone	Model in the paper	Training
tinygsm/sfm/sphere_arch_truncated_adaptive_no_renorm_cont.ckpt	TinyGSM → GSM8K	S-arch	Cont.	released S-FLM + 2,000 updates, original objective
tinygsm/sfm/sphere_arch_truncated_adaptive_no_renorm_tft.ckpt	TinyGSM → GSM8K	S-arch	TFT	released S-FLM + 2,000 updates, tempered-flow training (temperature-conditioned)
tinygsm/sfm/sphere_dit_truncated_adaptive_no_renorm_cont.ckpt	TinyGSM → GSM8K	DiT	Cont.	released S-FLM + 2,000 updates, original objective
tinygsm/sfm/sphere_dit_truncated_adaptive_no_renorm_tft.ckpt	TinyGSM → GSM8K	DiT	TFT	released S-FLM + 2,000 updates, tempered-flow training (temperature-conditioned)
sudoku_hard/sfm/sfm_truncated_adaptive.ckpt	Sudoku-Hard	DiT (tiny)	S-FLM	trained from scratch with the released S-FLM Sudoku recipe, 20,000 steps (no Sudoku checkpoint is released)
sudoku_hard/sfm/sfm_truncated_adaptive_cont.ckpt	Sudoku-Hard	DiT (tiny)	Cont.	the model above + 2,000 updates, original objective
sudoku_hard/sfm/sfm_truncated_adaptive_tft.ckpt	Sudoku-Hard	DiT (tiny)	TFT	the model above + 2,000 updates, tempered-flow training (temperature-conditioned)

File names follow those of the released S-FLM checkpoints, with a _cont / _tft suffix. Cont. and TFT of the same backbone start from the same checkpoint and use the same number of updates, so that "Cont. vs. TFT" isolates the training objective. TFT checkpoints contain additional temperature-conditioning parameters and must be evaluated with model.temp_conditioning=True.

SHA-256: see the "Files and versions" tab (LFS object IDs).

Baselines (not included; download them yourself)

Model	Location
S-FLM, S-arch (TinyGSM)	jdeschena/s-flm → tinygsm/sfm/sphere_arch_truncated_adaptive_no_renorm.ckpt
S-FLM, DiT (TinyGSM)	jdeschena/s-flm → tinygsm/sfm/sphere_dit_truncated_adaptive_no_renorm.ckpt
MDLM (TinyGSM)	jdeschena/s-flm → tinygsm/mdlm.ckpt
Duo (TinyGSM)	jdeschena/s-flm → tinygsm/duo.ckpt

from huggingface_hub import hf_hub_download

# ours
ckpt = hf_hub_download("daisuk30ba/BCT", "tinygsm/sfm/sphere_arch_truncated_adaptive_no_renorm_tft.ckpt")
# baseline (released S-FLM)
base = hf_hub_download("jdeschena/s-flm", "tinygsm/sfm/sphere_arch_truncated_adaptive_no_renorm.ckpt")

Format

Full PyTorch Lightning checkpoints in the same format as the released S-FLM checkpoints (model weights, EMA weights, optimizer and LR-scheduler state, adaptive noise-schedule state, and the training config), so they can be evaluated with the same loader and training can be resumed. They contain pickled Python objects (config and tokenizer); load them only with code you trust (weights_only=False). Evaluation uses the EMA weights, as in S-FLM.

Machine-specific paths stored in the checkpoints were replaced with /path/to/bct/..., and the stored training config uses the option names of the released code. Strings inherited from the released S-FLM checkpoints were left unchanged. Weights and all other tensors are bit-identical to the checkpoints used in the paper.

Training data

• TinyGSM (TinyGSM/TinyGSM, MIT), with the S-FLM preprocessing.
• Sudoku-Hard puzzles generated with the S-FLM Sudoku generator (official seed and split).

License

Apache-2.0. The models are derived from S-FLM (Apache-2.0).

Citation

@article{oba2026bct,
  title={BCT: Bayes-Consistent Tempering for Flow Language Models},
  author={Oba, Daisuke and Furuta, Hiroki and Okazaki, Naoaki},
  journal={arXiv preprint arXiv:2610.27641},
  year={2026}
}