Maggio33 commited on
Commit
b57ec2a
·
verified ·
1 Parent(s): b25aec4

Card update: training resumed 14 Nov from update 40,000 (agents 2fb783b0)

Browse files
Files changed (1) hide show
  1. TRAINING_CONFIGURATION.md +4 -2
TRAINING_CONFIGURATION.md CHANGED
@@ -57,7 +57,9 @@ Inherited-count footprint: counter SHA256 `abb39b8ce019c1a18a19ccf0ef4c32977c0c5
57
 
58
  ## Current CPT execution receipt
59
 
60
- **Latest execution receipt — 14 November 2026, 01:50 UTC:** operators confirmed continued pretraining resumed from **update 40,000** on **2 × NVIDIA RTX 4090**. Training updates from **40,001** retain the same effective batch of **256 sequences × 4,096 tokens per update**. Training is not finished. **Archive availability, 14 November 2026:** eight weights-only checkpoints cover updates 5,000–40,000; the published full-state recovery checkpoint remains at update 35,000. This availability does not extend benchmark coverage beyond update 35,000.
 
 
61
 
62
  **Historical launch receipt — 10 November 2026:** operators confirmed the 04:50 UTC start: fresh 1.07 hardware control within the 0.001 tolerance, pinned 1.08 weights/config converted to the training format, step-zero `INIT-FROM`, fresh optimizer, world size eight, c01/queue/CL validation identities and `PRE-FLIGHT OK`. The inherited 1.08 checkpoint step 1,709 is source metadata, **not the new continuation's initial training counter**.
63
 
@@ -75,7 +77,7 @@ The distinct [CPT Track run](https://track.fabryka.ai/run/71cf98ec-7c60-54a8-ac4
75
 
76
  The older public `train_gpt_ref_r23.py` is **not the identity of this training implementation**. The inference-compatible architecture code packaged with a weight release and the executable used to train it are distinct artifacts; filenames alone do not establish identity.
77
 
78
- The archived B200 recipe uses one eight-GPU node, PyTorch 2.8/CUDA 12.8 and DDP. It is a historical configuration reference, not a description of the 14 November two-RTX-4090 runtime. Compute uses bf16 on the verified hardware; `--dtype uint16` below describes **stored token IDs**, not model arithmetic.
79
 
80
  ## Historical shared learning arguments
81
 
 
57
 
58
  ## Current CPT execution receipt
59
 
60
+ **Execution receipt — 15 November 2026, 20:02 UTC:** operators confirmed continued pretraining resumed from the full checkpoint at **update 42,800** on **8 × NVIDIA B300** (spot instance), with the same trainer and the same effective batch of **256 sequences × 4,096 tokens per update**. On the 8 logged points from update 42,810 to 42,880, the training loss matched the earlier RTX 4090 run within 0.0003 (mean difference −0.0002). Training is not finished.
61
+
62
+ **Execution receipt — 14 November 2026, 01:50 UTC:** operators confirmed continued pretraining resumed from **update 40,000** on **2 × NVIDIA RTX 4090**. Training updates from **40,001** retain the same effective batch of **256 sequences × 4,096 tokens per update**. Training is not finished. **Archive availability, 14 November 2026:** eight weights-only checkpoints cover updates 5,000–40,000; the published full-state recovery checkpoint remains at update 35,000. This availability does not extend benchmark coverage beyond update 35,000.
63
 
64
  **Historical launch receipt — 10 November 2026:** operators confirmed the 04:50 UTC start: fresh 1.07 hardware control within the 0.001 tolerance, pinned 1.08 weights/config converted to the training format, step-zero `INIT-FROM`, fresh optimizer, world size eight, c01/queue/CL validation identities and `PRE-FLIGHT OK`. The inherited 1.08 checkpoint step 1,709 is source metadata, **not the new continuation's initial training counter**.
65
 
 
77
 
78
  The older public `train_gpt_ref_r23.py` is **not the identity of this training implementation**. The inference-compatible architecture code packaged with a weight release and the executable used to train it are distinct artifacts; filenames alone do not establish identity.
79
 
80
+ The archived B200 recipe uses one eight-GPU node, PyTorch 2.8/CUDA 12.8 and DDP. It is a historical configuration reference, not a description of the later runtimes (two RTX 4090 from 14 November, eight B300 from 15 November). Compute uses bf16 on the verified hardware; `--dtype uint16` below describes **stored token IDs**, not model arithmetic.
81
 
82
  ## Historical shared learning arguments
83