japhba's picture
Single-token-per-step latent-CoT organism: load-bearing + length-generalising
74fd26a verified
|
Raw
History Blame Contribute Delete
968 Bytes
# training_code β€” single-token-per-step latent-CoT organism
- `latent_threads/single.py` β€” single-token Markov mask, the T-step in-graph forward with codebook
feedback, readouts (one z_t per step; contrast latent_threads/markov.py).
- `latent_threads/train_single.py` β€” trainer (answer CE + per-cell feedback CE, teacher-forcing
anneal, mixed chain lengths for length generalisation, separable codebook init).
- `latent_threads/eval_single_report.py` β€” causal load-bearing battery + length-generalisation curve.
- `latent_threads/eval_single_cellpatch.py` β€” single-cell `c_i(t):=d` patch + CA-propagation check.
- `latent_threads/plot_single_summary.py` β€” training curve (readout/state/curriculum vs TF anneal).
- `latent_threads/tasks.py` β€” the diffuse coupled-CA task. Deps: abstract_cot/masking.py,
model_organisms/envs/base.py.
Retrain: `python -m latent_threads.train_single --config latent_threads/configs/single_k3m6.json --batch-id sg1`.