| # Artifacts |
|
|
| This release is meant to tell the short artifact story first, with the |
| technical details in manifests and provenance. |
|
|
| ```text |
| swiss-ai/Apertus-8B-2509 |
| | |
| | continuous BPE on GlossAPI + HPLT 50/50 |
| v |
| C3 candidate tokenizer |
| base 131072 + 25600 = 156672 |
| | |
| | cutoff sweep, curation, backfill, alignment |
| v |
| ModernGreek-148k |
| base 131072 + modern Greek 17408 = 148480 |
| | |
| | optional polytonic stack |
| v |
| ModernGreek-Polytonic-154k |
| base 131072 + modern Greek 17408 + polytonic Greek 5120 = 153600 |
| |
| fffoivos/glossapi-greek-nanochat-pretraining-dataset |
| + nanochat internal dedup metadata |
| + Apertus overlap drop overlay |
| | |
| v |
| selected_after_apertus_and_internal_dedup.parquet |
| | |
| | 70/24/4/2 Greek/replay/code/math mix |
| v |
| CPT-7B-mix |
| | |
| | token-count audit with ModernGreek-148k |
| | HPLT clean60 Wave4: 48,728,774 rows -> 44,195,950,025 tokens |
| | |
| | Megatron indexed dataset preprocessing |
| v |
| bulk_mix_base_nfc_megatron/bulk_mix_text_document.{bin,idx} |
| |
| ModernGreek-148k + CPT-7B-mix |
| | |
| | Vanilla, ReTok, Centroid, TokenDistil |
| v |
| 4-arm init bakeoff |
| | |
| +-> 2B endpoint (all 4 arms) |
| | -> 2B-iso checkpoints + iter-476 evals |
| | |
| +-> 3.5B continuation (Vanilla / ReTok / TokenDistil) |
| | -> iso-token checkpoints + benchmark-evals/3.5B-comparison |
| | |
| +-> 5B continuation (Vanilla / TokenDistil only) |
| -> bakeoff-final endpoint |
| -> Vanilla-5B (BPB leader) + TokenDistil-5B (downstream leader) |
| -> benchmark-evals/bakeoff-final |
| -> provenance/decisions/CPT_MASTER_20260526.md (synthesis) |
| -> provenance/decisions/PLAN_VS_RESULTS_RECONCILIATION_20260526.md |
| ``` |
|
|
| The pre-commit decision-rule thresholds from the v0.12 experimental plan (`old_experiments_plan.md` §10 Q8: X / M_progress / M_ext / M_van / T) were never locked before bakeoff results came in. The 5B headline is therefore an honest description of the numbers — TokenDistil-5B wins downstream, Vanilla-5B wins tokenizer-fair BPB — not a rule-bound winner. See `provenance/decisions/PLAN_VS_RESULTS_RECONCILIATION_20260526.md` for the 14-entry discrepancy log. |
| |
| `experiment-checkpoints/` is the top-level checkpoint area. It contains one |
| folder per public experiment checkpoint; source paths and exact technical |
| details live in each checkpoint manifest. |
| |
| `cpt-training-dataset/token-counts.json` records the source-token accounting: |
| the full staged `HPLT/ell_Grek_ge8_no_mt_clean60` slice tokenizes to |
| `44,195,950,025` `ModernGreek-148k` tokens without document separators, or |
| `44,244,678,799` tokens when adding one EOD per row. |
|
|
| Loss evidence follows the tokenizer-fair policy in |
| `provenance/evals/LOSS_MEASUREMENT_POLICY.md`: heldout BPB and downstream evals |
| decide cross-arm comparisons. Raw Megatron `lm loss` is a training health trace |
| because the compared arms do not all use the same tokenizer. Older files may |
| call BPB `BPC`; that is a legacy bits-per-byte label. |
|
|
| Checkpoint weights were uploaded from Clariden through the non-GPU `xfer` |
| partition: |
|
|
| ```text |
| Slurm job: 2382635 |
| state: COMPLETED |
| exit code: 0:0 |
| log: /users/fffoivos/apertus_hf_upload_checkpoints_20260525_2382635.log |
| ``` |
|
|
