Upload README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: license
|
| 4 |
+
license_link: LICENSE
|
| 5 |
+
---
|
| 6 |
+
<div align="center">
|
| 7 |
+
<h1>
|
| 8 |
+
Index-1.9B-Constant-LR
|
| 9 |
+
</h1>
|
| 10 |
+
</div>
|
| 11 |
+
|
| 12 |
+
## Model Introduction
|
| 13 |
+
This repository Index-1.9B-Constant-LR is the checkpoint file of the [Index-1.9B](https://huggingface.co/IndexTeam/Index-1.9B) base model before decay training, which is provided for everyone to conduct research on downstream tasks.
|
| 14 |
+
|
| 15 |
+
For more details, see our [GitHub](https://github.com/bilibili/Index-1.9B) and [Index-1.9B Technical Report](https://github.com/bilibili/Index-1.9B/blob/main/Index-1.9B%20%E6%8A%80%E6%9C%AF%E6%8A%A5%E5%91%8A.pdf)
|
| 16 |
+
|
| 17 |
+
## Evaluation Results
|
| 18 |
+
Here we add the evaluation of the general understanding ability of the Index-1.9B-Constant-LR model
|
| 19 |
+
|Model|Average score|Average English score|MMLU|CEVAL|CMMLU|HellaSwag|Arc-C|Arc-E|
|
| 20 |
+
|----|----|----|----|----|----|----|----|----|
|
| 21 |
+
|**Index-1.9B-Constant-LR**|41.47 |44.24 |35.30|38.58|33.26|59.94|32.96|48.75|
|
| 22 |
+
|**Index-1.9B-Pure**|49.55 |52.83 |43.75|42.35|43.61|63.21|42.75|61.61|
|
| 23 |
+
|**Index-1.9B**|**64.92** |**69.93**|52.53|57.01|52.79|80.69|65.15|81.35|
|
| 24 |
+
|
| 25 |
+
Evaluation code is based on [OpenCompass](https://github.com/open-compass/opencompass) with compatibility modifications. See the [evaluate](./evaluate/) folder for details.
|