japhba's picture
Single-token-per-step latent-CoT organism: load-bearing + length-generalising
74fd26a verified
|
Raw
History Blame Contribute Delete
968 Bytes

training_code — single-token-per-step latent-CoT organism

  • latent_threads/single.py — single-token Markov mask, the T-step in-graph forward with codebook feedback, readouts (one z_t per step; contrast latent_threads/markov.py).
  • latent_threads/train_single.py — trainer (answer CE + per-cell feedback CE, teacher-forcing anneal, mixed chain lengths for length generalisation, separable codebook init).
  • latent_threads/eval_single_report.py — causal load-bearing battery + length-generalisation curve.
  • latent_threads/eval_single_cellpatch.py — single-cell c_i(t):=d patch + CA-propagation check.
  • latent_threads/plot_single_summary.py — training curve (readout/state/curriculum vs TF anneal).
  • latent_threads/tasks.py — the diffuse coupled-CA task. Deps: abstract_cot/masking.py, model_organisms/envs/base.py.

Retrain: python -m latent_threads.train_single --config latent_threads/configs/single_k3m6.json --batch-id sg1.