Slayer1B 1.07: training-time evaluation at step 18590

README.md

@@ -21,7 +21,7 @@ Slayer1B 1.06** from its final checkpoint (step 13848) on 8 891 005 707 training
 replayed from the 1.06 data), at the same constant learning rate and global batch.
 
 **Status: training in progress.** The planned end is step 22327. This card reports intermediate training-time
-evaluations up to step 18000 and will be replaced by the final card when training ends. No 1.07 weights are
+evaluations up to step 18590 and will be replaced by the final card when training ends. No 1.07 weights are
 published in this repository yet.
 
 - Previous version: [SlayerLab/Slayer1B-1.06](https://huggingface.co/SlayerLab/Slayer1B-1.06) (step 13848,
@@ -77,18 +77,19 @@ ARC-Easy-PL eu20). 1.07 checkpoints are evaluated on CPU; the 1.06 checkpoint (s
 setup matches the 1.06 card values at card rounding. These are **not** the lm-evaluation-harness benchmark used in the
 Results table of 1.00–1.03.
 
-| metric | 1.06 (13848) | 13998 | 14000 | 15000 | 15730 | 16000 | 17000 | **18000** |
-|---|---:|---:|---:|---:|---:|---:|---:|---:|
-| BLiMP (EN), accuracy % | 76.12 | 76.29 | 76.33 | 77.94 | 77.32 | 77.13 | 77.61 | **77.88** |
-| ARC-Easy (EN), accuracy % | 52.99 | 54.34 | 54.21 | 54.59 | 55.64 | 56.73 | 56.48 | **56.52** |
-| WikiText-2, byte perplexity ↓ | 2.2849 | 2.2618 | 2.2601 | 2.2321 | 2.2237 | 2.2205 | 2.2138 | **2.2042** |
-| WikiText-2, token perplexity ↓ | 30.03 | 28.80 | 28.71 | 27.28 | 26.85 | 26.70 | 26.37 | **25.90** |
-| MultiBLiMP (PL), accuracy % | 98.41 | 98.66 | 98.59 | 98.62 | 98.62 | 98.81 | 98.87 | **98.84** |
-| ARC-Easy (PL, eu20), accuracy % | 45.03 | 45.37 | 44.91 | 45.92 | 45.75 | 46.25 | 45.41 | **46.21** |
+| metric | 1.06 (13848) | 13998 | 14000 | 15000 | 15730 | 16000 | 17000 | 18000 | **18590** |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| BLiMP (EN), accuracy % | 76.12 | 76.29 | 76.33 | 77.94 | 77.32 | 77.13 | 77.61 | 77.88 | **77.46** |
+| ARC-Easy (EN), accuracy % | 52.99 | 54.34 | 54.21 | 54.59 | 55.64 | 56.73 | 56.48 | 56.52 | **56.52** |
+| WikiText-2, byte perplexity ↓ | 2.2849 | 2.2618 | 2.2601 | 2.2321 | 2.2237 | 2.2205 | 2.2138 | 2.2042 | **2.2145** |
+| WikiText-2, token perplexity ↓ | 30.03 | 28.80 | 28.71 | 27.28 | 26.85 | 26.70 | 26.37 | 25.90 | **26.40** |
+| MultiBLiMP (PL), accuracy % | 98.41 | 98.66 | 98.59 | 98.62 | 98.62 | 98.81 | 98.87 | 98.84 | **98.62** |
+| ARC-Easy (PL, eu20), accuracy % | 45.03 | 45.37 | 44.91 | 45.92 | 45.75 | 46.25 | 45.41 | 46.21 | **47.14** |
 
 ARC-Easy has 2 376 questions per language, so differences below about one percentage point between neighbouring
-checkpoints are within noise. The training loss in the figure is measured on the 1.07 data mixture (44 % English),
-so it is not directly comparable with the training loss of 1.00–1.06.
+checkpoints are within noise. WikiText-2 byte perplexity at 18590 is 0.010 higher than at 18000; it is a single
+intermediate point of a constant-learning-rate run, reported as measured. The training loss in the figure is measured
+on the 1.07 data mixture (44 % English), so it is not directly comparable with the training loss of 1.00–1.06.
 
 The benchmark caveat of the 1.03 card applies to the 1.00–1.06 part of the training. For 1.07, all nine data files
 were scanned for 13-gram overlap with the test items of the evaluation suite before training, and documents with any