Time-locked English 1780-1830 (590M, QK-norm)
A GPT-style model trained from scratch only on English printed 1780-1830: 28 layers, width 1280, 20 heads, QK-norm (each head's queries and keys layer-normalised, Amendment 39), 592,849,664 parameters, 48,931 steps of 524,288 tokens (one pass over 25.7B tokens), peak learning rate 2.5e-4, bf16 on two H100s. Final validation loss 2.1917 at step 48,931 (the 297M reached 2.3248).
QK-norm is not in the GPT-2 class, so the weights are native PyTorch with their own loader:
from modeling_timelocked import load # this repo's file
from tokenizers import Tokenizer
model, tok = load(), Tokenizer.from_file("tokenizer.json")
model.pt is the training run's own state dict; model.safetensors the same weights;
milestones/stepNNNNNN_model.pt the weights every 5,000 steps; loss_curves.csv every step.
Benchmark (110 held-out tests, data/benchmark/alm_mid.json in the GitHub repository): the
primary method picks the right author 0.982 of the time with 0.218 false matches.
The study and what it found
This is one of the models from a finished, open study of the English text of the Book of Mormon: who writes like it, what it borrows, and when its language dates from.
- The findings in plain words, with live charts: https://lifehug.com/book-of-mormon
- The paper, the code and every results file: https://github.com/lifehug/book-of-mormon-authorship
In short: the book copies almost nothing from earlier print. Among the eight people usually proposed as its author, the closest writing is Joseph Smith's revelations, and a search of 23,872 authors of the period puts those revelations first or second. No tool we found, ours included, can tell a person from the voice they are writing in, or reliably say "it is none of these people". The study takes no position on inspiration.
About
Part of an open, pre-registered study of the authorship, borrowing and dating of the
earliest Book of Mormon text (https://github.com/lifehug/book-of-mormon-authorship; pre-registration https://osf.io/6ugxc/). The project does not argue for or
against divine inspiration. Everything is published so it can be checked and redone; the corpus
texts we agreed to keep private are not, and the repository's docs/replication.md gives the
steps to obtain the same access.
Licence: CC BY-NC 4.0. Training data: English printed 1780-1830 (Internet Archive, Harvard
Institutional Books 1.0, HathiTrust public-domain volumes, Text Creation Partnership), with the
Book of Mormon and everything after 1830 removed; see the base model cards
(etherfuse/timelocked-1830-125m). Contains derivatives of Harvard Institutional Books 1.0,
Institutional Data Initiative, Harvard Law School Library; used for noncommercial research.
Checksums: every file's SHA-256 is in the GitHub repository, data/manifests/hf_<repo>.jsonl.
- Downloads last month
- 42