Commit
Β·
164b617
1
Parent(s):
2f23fe4
Update README.md
Browse files
README.md
CHANGED
|
@@ -14,6 +14,15 @@ language:
|
|
| 14 |
- fr
|
| 15 |
- bn
|
| 16 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 17 |
### Introduction
|
| 18 |
|
| 19 |
We introduce π MathOctopus, a series of open-source large language models (LLMs) specifically tailored for multilingual math problem-solving. The MathOctopus models are trained on π€ MGSM8KInstruct Dataset, encompassing ten distinct languages.
|
|
@@ -61,7 +70,9 @@ Our dataset and models are all available at Huggingface.
|
|
| 61 |
*-Cross refers to our model trained with cross-training strategy.
|
| 62 |
|
| 63 |
*-xRFT means we train the model with multilingual rejection sampling.
|
|
|
|
| 64 |
### **Overall Results on MGSM**
|
|
|
|
| 65 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
| 66 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
| 67 |
| MathOctopus<sup>C</sup> | 52.0 | 23.6 | 31.6 | 18.8 | 38.0 | 39.2 | 36.4 | 27.2 | 33.6 | 21.6 | 32.2 |
|
|
@@ -85,7 +96,10 @@ Our dataset and models are all available at Huggingface.
|
|
| 85 |
| **xRFT**-MathOctopus<sup>C</sup>| 53.6 | 27.6 | 34.4 | 19.2 | 47.2 | 47.6 | 44.8 | 30.8 | 38.8 | 22.8 | 36.7 |
|
| 86 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
| 87 |
| **xRFT**-MathOctopus<sup>P</sup>| 51.6 | 47.2 | 52.4 | 37.6 | 51.2 | 52.8 | 44.4 | 41.6 | 50.0 | 47.6 | 47.6 |
|
|
|
|
|
|
|
| 88 |
### **Overall Results on MSVAMP**
|
|
|
|
| 89 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
| 90 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
| 91 |
| MathOctopus<sup>C</sup> | 49.2 | 36.6 | 43.6 | 30.2 | 48.6 | 46.8 | 46.4 | 42.5 | 46.7 | 34.0 | 42.5 |
|
|
@@ -109,7 +123,10 @@ Our dataset and models are all available at Huggingface.
|
|
| 109 |
| **xRFT**-MathOctopus<sup>C</sup>| 48.1 | 42.8 | 43.6 | 23.3 | 48.7 | 50.0 | 48.9 | 43.4 | 44.6 | 35.5 | 42.9 |
|
| 110 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
| 111 |
| **xRFT**-MathOctopus<sup>P</sup>| 48.0 | 42.3 | 46.1 | 36.2 | 47.5 | 48.5 | 48.3 | 45.8 | 47.2 | 41.2 | 45.1 |
|
|
|
|
|
|
|
| 112 |
### **MathOctopus in English**
|
|
|
|
| 113 |
| Models | GSM8K | SVAMP |
|
| 114 |
|:--------------------------------|:--------|:--------|
|
| 115 |
| LLaMA 2-7B | 42.4 | 38.3 |
|
|
@@ -121,5 +138,20 @@ Our dataset and models are all available at Huggingface.
|
|
| 121 |
| LLaMA 1-33B | 50.0 | 49.0 |
|
| 122 |
| MathOctopus<sup>P</sup>-33B | 56.0 | 52.5 |
|
| 123 |
| MathOctopus<sup>C</sup>-33B | 53.7 | 51.5 |
|
|
|
|
| 124 |
## Intended Uses
|
| 125 |
These models are trained for research purposes. They are designed to solve multilingual math problems. They can be used in educational software, tutoring systems, or any application where a solution to a math problem is needed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
- fr
|
| 15 |
- bn
|
| 16 |
---
|
| 17 |
+
|
| 18 |
+
# π Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations
|
| 19 |
+
|
| 20 |
+
Project Page: [https://mathoctopus.github.io/](https://mathoctopus.github.io/)
|
| 21 |
+
|
| 22 |
+
Paper: [https://arxiv.org/abs/2310.20246.pdf](https://arxiv.org/abs/2310.20246.pdf)
|
| 23 |
+
|
| 24 |
+
Code: [https://github.com/microsoft/MathOctopus](https://github.com/microsoft/MathOctopus)
|
| 25 |
+
|
| 26 |
### Introduction
|
| 27 |
|
| 28 |
We introduce π MathOctopus, a series of open-source large language models (LLMs) specifically tailored for multilingual math problem-solving. The MathOctopus models are trained on π€ MGSM8KInstruct Dataset, encompassing ten distinct languages.
|
|
|
|
| 70 |
*-Cross refers to our model trained with cross-training strategy.
|
| 71 |
|
| 72 |
*-xRFT means we train the model with multilingual rejection sampling.
|
| 73 |
+
|
| 74 |
### **Overall Results on MGSM**
|
| 75 |
+
|
| 76 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
| 77 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
| 78 |
| MathOctopus<sup>C</sup> | 52.0 | 23.6 | 31.6 | 18.8 | 38.0 | 39.2 | 36.4 | 27.2 | 33.6 | 21.6 | 32.2 |
|
|
|
|
| 96 |
| **xRFT**-MathOctopus<sup>C</sup>| 53.6 | 27.6 | 34.4 | 19.2 | 47.2 | 47.6 | 44.8 | 30.8 | 38.8 | 22.8 | 36.7 |
|
| 97 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
| 98 |
| **xRFT**-MathOctopus<sup>P</sup>| 51.6 | 47.2 | 52.4 | 37.6 | 51.2 | 52.8 | 44.4 | 41.6 | 50.0 | 47.6 | 47.6 |
|
| 99 |
+
|
| 100 |
+
|
| 101 |
### **Overall Results on MSVAMP**
|
| 102 |
+
|
| 103 |
| 7B Model | En | Sw | Zh | Bn | De | Es | Fr | Ja | Ru | Th | Overall |
|
| 104 |
|:--------------------------------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|:--------|
|
| 105 |
| MathOctopus<sup>C</sup> | 49.2 | 36.6 | 43.6 | 30.2 | 48.6 | 46.8 | 46.4 | 42.5 | 46.7 | 34.0 | 42.5 |
|
|
|
|
| 123 |
| **xRFT**-MathOctopus<sup>C</sup>| 48.1 | 42.8 | 43.6 | 23.3 | 48.7 | 50.0 | 48.9 | 43.4 | 44.6 | 35.5 | 42.9 |
|
| 124 |
| MathOctopus<sup>P</sup> | 56.4 | 46.8 | 52.0 | 35.2 | 47.2 | 53.2 | 48.0 | 39.2 | 45.6 | 41.2 | 46.5 |
|
| 125 |
| **xRFT**-MathOctopus<sup>P</sup>| 48.0 | 42.3 | 46.1 | 36.2 | 47.5 | 48.5 | 48.3 | 45.8 | 47.2 | 41.2 | 45.1 |
|
| 126 |
+
|
| 127 |
+
|
| 128 |
### **MathOctopus in English**
|
| 129 |
+
|
| 130 |
| Models | GSM8K | SVAMP |
|
| 131 |
|:--------------------------------|:--------|:--------|
|
| 132 |
| LLaMA 2-7B | 42.4 | 38.3 |
|
|
|
|
| 138 |
| LLaMA 1-33B | 50.0 | 49.0 |
|
| 139 |
| MathOctopus<sup>P</sup>-33B | 56.0 | 52.5 |
|
| 140 |
| MathOctopus<sup>C</sup>-33B | 53.7 | 51.5 |
|
| 141 |
+
|
| 142 |
## Intended Uses
|
| 143 |
These models are trained for research purposes. They are designed to solve multilingual math problems. They can be used in educational software, tutoring systems, or any application where a solution to a math problem is needed.
|
| 144 |
+
|
| 145 |
+
## Citation
|
| 146 |
+
Please cite our paper if you use our data, model or code. Please also kindly cite the original dataset papers.
|
| 147 |
+
|
| 148 |
+
```
|
| 149 |
+
@misc{chen2023breaking,
|
| 150 |
+
title={Breaking Language Barriers in Multilingual Mathematical Reasoning: Insights and Observations},
|
| 151 |
+
author={Nuo Chen and Zinan Zheng and Ning Wu and Linjun Shou and Ming Gong and Yangqiu Song and Dongmei Zhang and Jia Li},
|
| 152 |
+
year={2023},
|
| 153 |
+
eprint={2310.20246},
|
| 154 |
+
archivePrefix={arXiv},
|
| 155 |
+
primaryClass={cs.CL}
|
| 156 |
+
}
|
| 157 |
+
```
|