ceselder/cot-oracle-paper-ablation-ours-3layers

CoT Oracle Paper Ablation: Ours, 3 Layers

This repo contains the 3-layer version of the core CoT Oracle paper ablation: on-policy lens tasks, chunked ConvQA, FineWeb lens readouts, and classification, without LatentQA.

What This Checkpoint Is

• Base model: Qwen/Qwen3-8B
• Adapter format: PEFT LoRA
• Activation readout layers: [9, 18, 27]
• Task order: shuffled
• Seed: 42
• Planned budget: 50M input tokens
• Paper label: 18M logged training tokens

Exact Training Mixture

• On-policy futurelens: enabled, n: 30000
• On-policy pastlens: enabled, n: 30000
• chunked_convqa: enabled, n: -1 (all available examples)
• classification: enabled, n: 20000, datasets = sst2, ag_news, snli
• fineweb: enabled, n: 60000, variants = futurelens_fineweb,pastlens_fineweb
• latentqa: disabled
• All other tasks in configs/train.yaml: disabled

Notes

• This run stopped early; this repo contains the latest recoverable uploaded checkpoint from that run.
• The paper label follows the logged training-token count for the run rather than the planned 50M input-token budget.