Add pipeline tag, clarify license, add library and second citation
#1
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,11 +1,13 @@
|
|
| 1 |
---
|
| 2 |
-
license: llama3.1
|
| 3 |
base_model:
|
| 4 |
- meta-llama/Llama-3.1-8B
|
| 5 |
datasets:
|
| 6 |
- nvidia/OpenMathInstruct-2
|
| 7 |
language:
|
| 8 |
- en
|
|
|
|
|
|
|
|
|
|
| 9 |
tags:
|
| 10 |
- nvidia
|
| 11 |
- math
|
|
@@ -15,7 +17,7 @@ tags:
|
|
| 15 |
|
| 16 |
[NeMo](https://github.com/NVIDIA/NeMo) checkpoint for [OpenMath2-Llama3.1-8B](https://huggingface.co/nvidia/OpenMath2-Llama3.1-8B) is obtained by finetuning [Llama3.1-8B-Base](https://huggingface.co/meta-llama/Llama-3.1-8B) with [OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2).
|
| 17 |
|
| 18 |
-
The model outperforms [Llama3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on all the popular math benchmarks we evaluate on, especially on [MATH](https://github.com/hendrycks/math) by 15.9%.
|
| 19 |
|
| 20 |
<!-- <p align="center">
|
| 21 |
<img src="scaling_plot.jpg" width="350"><img src="math_level_comp.jpg" width="350">
|
|
@@ -58,12 +60,12 @@ See our [paper](https://arxiv.org/abs/2410.01560) to learn more details!
|
|
| 58 |
|
| 59 |
# How to use the models?
|
| 60 |
|
| 61 |
-
Our models are trained with the same "chat format" as Llama3.1-instruct models (same system/user/assistant tokens).
|
| 62 |
Please note that these models have not been instruction tuned on general data and thus might not provide good answers outside of math domain.
|
| 63 |
|
| 64 |
-
This is a NeMo checkpoint, so you need to use [NeMo Framework](https://github.com/NVIDIA/NeMo) to run inference or finetune it.
|
| 65 |
We also release a [HuggingFace checkpoint](https://huggingface.co/nvidia/OpenMath2-Llama3.1-8B) and provide easy instructions on how to
|
| 66 |
-
[convert between different formats](https://nvidia.github.io/NeMo-Skills/pipelines/checkpoint-conversion/) or
|
| 67 |
[run inference](https://nvidia.github.io/NeMo-Skills/basics/inference/) with these models using our codebase.
|
| 68 |
|
| 69 |
# Reproducing our results
|
|
@@ -75,7 +77,7 @@ We provide [all instructions](https://nvidia.github.io/NeMo-Skills/openmathinstr
|
|
| 75 |
If you find our work useful, please consider citing us!
|
| 76 |
|
| 77 |
```bibtex
|
| 78 |
-
@article{
|
| 79 |
title = {OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data},
|
| 80 |
author = {Shubham Toshniwal and Wei Du and Ivan Moshkov and Branislav Kisacanin and Alexan Ayrapetyan and Igor Gitman},
|
| 81 |
year = {2024},
|
|
@@ -83,6 +85,13 @@ If you find our work useful, please consider citing us!
|
|
| 83 |
}
|
| 84 |
```
|
| 85 |
|
| 86 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 87 |
|
| 88 |
-
|
|
|
|
| 1 |
---
|
|
|
|
| 2 |
base_model:
|
| 3 |
- meta-llama/Llama-3.1-8B
|
| 4 |
datasets:
|
| 5 |
- nvidia/OpenMathInstruct-2
|
| 6 |
language:
|
| 7 |
- en
|
| 8 |
+
license: mit
|
| 9 |
+
pipeline_tag: text-generation
|
| 10 |
+
library_name: transformers
|
| 11 |
tags:
|
| 12 |
- nvidia
|
| 13 |
- math
|
|
|
|
| 17 |
|
| 18 |
[NeMo](https://github.com/NVIDIA/NeMo) checkpoint for [OpenMath2-Llama3.1-8B](https://huggingface.co/nvidia/OpenMath2-Llama3.1-8B) is obtained by finetuning [Llama3.1-8B-Base](https://huggingface.co/meta-llama/Llama-3.1-8B) with [OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2).
|
| 19 |
|
| 20 |
+
The model outperforms [Llama3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on all the popular math benchmarks we evaluate on, especially on [MATH](https://github.com/hendrycks/math) by 15.9%.
|
| 21 |
|
| 22 |
<!-- <p align="center">
|
| 23 |
<img src="scaling_plot.jpg" width="350"><img src="math_level_comp.jpg" width="350">
|
|
|
|
| 60 |
|
| 61 |
# How to use the models?
|
| 62 |
|
| 63 |
+
Our models are trained with the same "chat format" as Llama3.1-instruct models (same system/user/assistant tokens).
|
| 64 |
Please note that these models have not been instruction tuned on general data and thus might not provide good answers outside of math domain.
|
| 65 |
|
| 66 |
+
This is a NeMo checkpoint, so you need to use [NeMo Framework](https://github.com/NVIDIA/NeMo) to run inference or finetune it.
|
| 67 |
We also release a [HuggingFace checkpoint](https://huggingface.co/nvidia/OpenMath2-Llama3.1-8B) and provide easy instructions on how to
|
| 68 |
+
[convert between different formats](https://nvidia.github.io/NeMo-Skills/pipelines/checkpoint-conversion/) or
|
| 69 |
[run inference](https://nvidia.github.io/NeMo-Skills/basics/inference/) with these models using our codebase.
|
| 70 |
|
| 71 |
# Reproducing our results
|
|
|
|
| 77 |
If you find our work useful, please consider citing us!
|
| 78 |
|
| 79 |
```bibtex
|
| 80 |
+
@article{toshniwal2024openmathinstruct2,
|
| 81 |
title = {OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Data},
|
| 82 |
author = {Shubham Toshniwal and Wei Du and Ivan Moshkov and Branislav Kisacanin and Alexan Ayrapetyan and Igor Gitman},
|
| 83 |
year = {2024},
|
|
|
|
| 85 |
}
|
| 86 |
```
|
| 87 |
|
| 88 |
+
```bibtex
|
| 89 |
+
@inproceedings{toshniwal2024openmathinstruct1,
|
| 90 |
+
title = {{OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset}},
|
| 91 |
+
author = {Shubham Toshniwal and Ivan Moshkov and Sean Narenthiran and Daria Gitman and Fei Jia and Igor Gitman},
|
| 92 |
+
year = {2024},
|
| 93 |
+
booktitle = {Advances in Neural Information Processing Systems},
|
| 94 |
+
}
|
| 95 |
+
```
|
| 96 |
|
| 97 |
+
Disclaimer: This project is strictly for research purposes, and not an official product from NVIDIA.
|