Fix the byte perplexity range sentence (from step 21000 on)
Browse files
README.md
CHANGED
|
@@ -100,7 +100,7 @@ lm-evaluation-harness benchmark used in the Results table of 1.00–1.03.
|
|
| 100 |
| **22327 (final)** | **77.90** | **58.67** | **2.1837** | **24.92** | **98.84** | **46.93** | CPU |
|
| 101 |
|
| 102 |
ARC-Easy has 2 376 questions per language, so differences below about one percentage point between neighbouring
|
| 103 |
-
checkpoints are within noise. From
|
| 104 |
falling steadily (lowest 2.1794 at 22000; 2.1837 at the final step): the run used a constant learning rate without decay,
|
| 105 |
so the last checkpoint is where training stopped, not a converged minimum. The final checkpoint has the highest English
|
| 106 |
ARC-Easy accuracy of the run (58.67, +2.0 points over 19000). Each point is a single measurement. The training loss in the figure is measured
|
|
|
|
| 100 |
| **22327 (final)** | **77.90** | **58.67** | **2.1837** | **24.92** | **98.84** | **46.93** | CPU |
|
| 101 |
|
| 102 |
ARC-Easy has 2 376 questions per language, so differences below about one percentage point between neighbouring
|
| 103 |
+
checkpoints are within noise. From 21000 on, WikiText-2 byte perplexity stays between 2.179 and 2.187 instead of
|
| 104 |
falling steadily (lowest 2.1794 at 22000; 2.1837 at the final step): the run used a constant learning rate without decay,
|
| 105 |
so the last checkpoint is where training stopped, not a converged minimum. The final checkpoint has the highest English
|
| 106 |
ARC-Easy accuracy of the run (58.67, +2.0 points over 19000). Each point is a single measurement. The training loss in the figure is measured
|