Maggio33 commited on
Commit
43e7ca2
·
verified ·
1 Parent(s): b7192e9

Card: BLiMP count note — cause of the 47 913 count not determined

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -29,7 +29,7 @@ weights via `load_slayer.py`, and the trainer checkpoint directly); both gave th
29
 
30
  | task | metric | n | score |
31
  |---|---|---:|---:|
32
- | BLiMP (67 phenomena) | accuracy | 67 000 | 71.51 % (47 912 correct; one evaluation on a different GPU counted 47 913, one pair flipping at a numerical tie) |
33
  | ARC-Easy (test) | accuracy | 2 376 | 42.89 % (1 019 correct) |
34
  | WikiText-2 (raw, test) | token perplexity (this tokenizer) | — | 49.9734 |
35
  | WikiText-2 (raw, test) | byte perplexity | 1 292 008 bytes | 2.5857 |
 
29
 
30
  | task | metric | n | score |
31
  |---|---|---:|---:|
32
+ | BLiMP (67 phenomena) | accuracy | 67 000 | 71.51 % (47 912 correct; an evaluation on another GPU counted 47 913, cause not determined: no per-item record from that run) |
33
  | ARC-Easy (test) | accuracy | 2 376 | 42.89 % (1 019 correct) |
34
  | WikiText-2 (raw, test) | token perplexity (this tokenizer) | — | 49.9734 |
35
  | WikiText-2 (raw, test) | byte perplexity | 1 292 008 bytes | 2.5857 |