Add real 100M -> 300M scaling benchmarks to org card
Browse files
README.md
CHANGED
|
@@ -16,6 +16,17 @@ pinned: false
|
|
| 16 |
- [LUNA-300M](https://huggingface.co/ASTERIZER/LUNA-300M) — 4.5B-token pretrain (2.7× scale-up)
|
| 17 |
- LUNA-1B — in progress
|
| 18 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
**Tokenizer (multilingual BPE)**
|
| 20 |
- [LUNA-1B-Tokenizer](https://huggingface.co/ASTERIZER/LUNA-1B-Tokenizer) — 128K / 64K / 8K vocab
|
| 21 |
- **2.8× better compression** than GPT-NeoX, **67% fewer tokens** on Indic text (Kannada 7×, Telugu 6×, Tamil 5×)
|
|
|
|
| 16 |
- [LUNA-300M](https://huggingface.co/ASTERIZER/LUNA-300M) — 4.5B-token pretrain (2.7× scale-up)
|
| 17 |
- LUNA-1B — in progress
|
| 18 |
|
| 19 |
+
**Scaling benchmarks — 100M → 300M** *(latest base checkpoints, same 4.5B-token corpus, held-out ~100K-token eval: EN / KN / TE / TA / HI + code)*
|
| 20 |
+
|
| 21 |
+
| metric | LUNA-100M | LUNA-300M | gain |
|
| 22 |
+
|---|---|---|---|
|
| 23 |
+
| Eval loss | 3.88 | 3.81 | ↓1.6% |
|
| 24 |
+
| Perplexity — overall | 48.2 | 45.2 | ↓6.1% |
|
| 25 |
+
| Perplexity — English | 41.9 | 35.2 | ↓16.0% |
|
| 26 |
+
| Perplexity — code | 51.9 | 43.9 | ↓15.4% |
|
| 27 |
+
| Perplexity — Kannada | 15.7 | 13.5 | ↓13.8% |
|
| 28 |
+
| Inference (CPU fp32) | 3,303 tok/s | 1,222 tok/s | 2.7× params |
|
| 29 |
+
|
| 30 |
**Tokenizer (multilingual BPE)**
|
| 31 |
- [LUNA-1B-Tokenizer](https://huggingface.co/ASTERIZER/LUNA-1B-Tokenizer) — 128K / 64K / 8K vocab
|
| 32 |
- **2.8× better compression** than GPT-NeoX, **67% fewer tokens** on Indic text (Kannada 7×, Telugu 6×, Tamil 5×)
|