Card: Polish leaderboard columns (two independent implementations)
Browse files
README.md
CHANGED
|
@@ -35,6 +35,14 @@ weights via `load_slayer.py`, and the trainer checkpoint directly); both gave th
|
|
| 35 |
| WikiText-2 (raw, test) | byte perplexity | 1 292 008 bytes | 2.5857 |
|
| 36 |
| eff (leaderboard score from BLiMP, ARC-Easy and WikiText-2 byte perplexity) | leaderboard formula | — | 69.50 |
|
| 37 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
## Usage
|
| 39 |
|
| 40 |
The weights are a plain PyTorch state dict in safetensors; they are not a `transformers` AutoModel. The model class
|
|
|
|
| 35 |
| WikiText-2 (raw, test) | byte perplexity | 1 292 008 bytes | 2.5857 |
|
| 36 |
| eff (leaderboard score from BLiMP, ARC-Easy and WikiText-2 byte perplexity) | leaderboard formula | — | 69.50 |
|
| 37 |
|
| 38 |
+
Polish columns of the same leaderboard (also computed twice with two independent implementations; identical item by
|
| 39 |
+
item):
|
| 40 |
+
|
| 41 |
+
| task | metric | n | score |
|
| 42 |
+
|---|---|---:|---:|
|
| 43 |
+
| MultiBLiMP, Polish (`jumelet/multiblimp` pol) | accuracy, summed log-probability, leading-space prefix | 3 272 | 97.89 % (3 203 correct) |
|
| 44 |
+
| ARC-Easy-PL (`Eurolingua/arcx` EU20, test) | accuracy, LL(q + choice) − LL(q) | 2 376 | 36.91 % (877 correct) |
|
| 45 |
+
|
| 46 |
## Usage
|
| 47 |
|
| 48 |
The weights are a plain PyTorch state dict in safetensors; they are not a `transformers` AutoModel. The model class
|