hamlab-princeton/pastcasting-talkie-cpt

Talkie: continuous historical CPT

Current run complete · updated November 2, 2026. This is FP32 full-parameter continual pretraining from Talkie 1930 Base, with one chronological pass in each new period from 1931–35 through 1968–72. All 16 half/full-pass checkpoints are archived, and Base plus all 16 checkpoints have complete raw-model factual evaluations on the same 52,000-question bank.

Factual accuracy by period

Full-size image · PDF · Period CSV · Yearly CSV · Complete evaluation evidence

On 1968–72 questions, factual accuracy is 55.10% for Base, 68.54% before this period's CPT, 74.54% at half pass, and 74.74% at full pass. These values measure the fixed factual bank. Training-source acquisition and the proposed 95% checkpoint-over-checkpoint knowledge-retention gate require separate evidence and are not certified by this plot.

Continuous CPT checkpoints

Each period continues from the preceding period's full-pass checkpoint. Half/full pass refers to exposure within the newest period; earlier periods have already received a full pass. All listed checkpoint folders contain sharded FP32 model weights.

New training period	Half pass	Full pass	Factual evaluation
1931–1935	Step 399	Step 797	52,000 each
1936–1941	Step 1,208	Step 1,619	52,000 each
1942–1947	Step 2,095	Step 2,570	52,000 each
1948–1952	Step 3,029	Step 3,487	52,000 each
1953–1957	Step 4,016	Step 4,545	52,000 each
1958–1962	Step 5,024	Step 5,503	52,000 each
1963–1967	Step 5,950	Step 6,397	52,000 each
1968–1972	Step 6,808	Step 7,219	52,000 each

The latest evaluated checkpoint is 1968–72 full pass, step 7,219. Machine-readable release status and evaluation results identify the frozen protocol and score coverage.

Evaluation dataset and interpretation

The frozen factual bank has 1,000 questions per year from 1921–1972. Each year has 250 people, 250 places, 250 organizations, and 250 numerical facts; every item has four answer choices with matched Talkie token lengths. Scores use FP32 candidate likelihoods on raw models, with no SFT or calibration. Equal question counts and the bank's difficulty matching do not prove equal difficulty for every model.

The archive includes score provenance, per-checkpoint result summaries, and the final evaluation's frozen launch and validation receipts. Forecasting questions and Brier-score experiments are separate from this factual bank.

Training recipe

• One full chronological pass per new period, with a half-pass checkpoint.
• 25% broad, year-balanced replay from eligible earlier years, expanding after each period.
• 64 sequence slots per update, each 1,024 tokens: 48 new-period and 16 replay slots, using gradient accumulation.
• Learning rate 1e-5, flat after 16 global warmup updates; no KL penalty.
• FP32 full-parameter training, with no LoRA. Evaluations and archive uploads run separately.

Run identity: pastcasting-continuous-flat-v3-20261022. Training manifest: 6a562cb6b7d2f91d2bb51b30dd67dd3e65e62c88fbab8da92629734709654409.

Download the final checkpoint

from pathlib import Path
from huggingface_hub import snapshot_download

subfolder = "cpt-v2/pastcasting-continuous-flat-v3-20261022/1968-1972/full-pass"
root = Path(snapshot_download(
    "hamlab-princeton/pastcasting-talkie-cpt",
    allow_patterns=[subfolder + "/*", "runtime/*", "base/talkie-1930/vocab.txt"],
))
checkpoint = root / subfolder

The repository is a checkpoint collection, not a single Transformers AutoModelForCausalLM target. CPT checkpoints use the custom Talkie loader in runtime/ and require substantial RAM. The original Base is at base/talkie-1930/final.ckpt.

Based on talkie-lm/talkie-1930-13b-base, revision b7c97680791f7fca4262c3c80b36ff7d666faab0. See NOTICE and LICENSE for upstream attribution.