Update README.md
Browse files
README.md
CHANGED
|
@@ -17,7 +17,6 @@ This repo contains the Base (completed pretrain cudalm checkpoint) variant of LS
|
|
| 17 |
| | HellaSwag | ARC-E | ARC-C | PIQA | LAMBADA | ArithMark | II* |
|
| 18 |
| ------------- | --------- | ----- | ----- | -------- | ------- | --------- | ---- |
|
| 19 |
| LS-Micro-Base | 27.41 | 34.72 | 21.93 | 54.84 | 11.06 | 35.50 | 7.24 |
|
| 20 |
-
| LS-Micro-CPT | 26.84 | 34.22 | 21.93 | 55.28 | 9.94 | 35.70 | 7.23 |
|
| 21 |
| LS-Tiny-Base | 27.11 | 32.24 | 24.49 | 53.81 | 2.99 | 34.00 | 6.22 |
|
| 22 |
|
| 23 |
LS-Micro is 3x smaller than LS-TIny (11M vs 31M), and has 3x less active parameters (5M vs 15M), yet has a 16% higher intelligence index.
|
|
@@ -53,7 +52,6 @@ Multiple choice:
|
|
| 53 |
| consistency | 18 | 61.1% | 72.2% | 25.0% |
|
| 54 |
| arithmetic | 15 | 46.7% | 46.7% | 25.0% |
|
| 55 |
| LS-Micro-Base | 118 | **46.6%** | **52.5%** | 25.0% |
|
| 56 |
-
| LS-Micro-CPT | 118 | 37.3% | 40.7% | 25.0% |
|
| 57 |
| LS-Tiny-Base | 118 | 40.7% | 47.5% | 25.0% |
|
| 58 |
|
| 59 |
## The base model is horrendous at chat, and is intended to be used as a base to be finetuned for specialised tasks.
|
|
|
|
| 17 |
| | HellaSwag | ARC-E | ARC-C | PIQA | LAMBADA | ArithMark | II* |
|
| 18 |
| ------------- | --------- | ----- | ----- | -------- | ------- | --------- | ---- |
|
| 19 |
| LS-Micro-Base | 27.41 | 34.72 | 21.93 | 54.84 | 11.06 | 35.50 | 7.24 |
|
|
|
|
| 20 |
| LS-Tiny-Base | 27.11 | 32.24 | 24.49 | 53.81 | 2.99 | 34.00 | 6.22 |
|
| 21 |
|
| 22 |
LS-Micro is 3x smaller than LS-TIny (11M vs 31M), and has 3x less active parameters (5M vs 15M), yet has a 16% higher intelligence index.
|
|
|
|
| 52 |
| consistency | 18 | 61.1% | 72.2% | 25.0% |
|
| 53 |
| arithmetic | 15 | 46.7% | 46.7% | 25.0% |
|
| 54 |
| LS-Micro-Base | 118 | **46.6%** | **52.5%** | 25.0% |
|
|
|
|
| 55 |
| LS-Tiny-Base | 118 | 40.7% | 47.5% | 25.0% |
|
| 56 |
|
| 57 |
## The base model is horrendous at chat, and is intended to be used as a base to be finetuned for specialised tasks.
|