Update README.md
Browse files
README.md
CHANGED
|
@@ -12,8 +12,8 @@ license: apache-2.0
|
|
| 12 |
<a href="https://huggingface.co/JT-LM/JT-Math-8B-Thinking" target="_blank">
|
| 13 |
<img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue">
|
| 14 |
</a>
|
| 15 |
-
|
| 16 |
-
<img
|
| 17 |
</a>
|
| 18 |
</p>
|
| 19 |
|
|
@@ -28,27 +28,22 @@ For full transparency and reproducibility, please refer to our technical report
|
|
| 28 |
|
| 29 |
|
| 30 |
|
|
|
|
| 31 |
|
|
|
|
| 32 |
|
| 33 |
|
| 34 |
-
**Figure 1: Performance of JT-Math-8B-Thinking on math reasoning benchmarks.**
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
## Model Highlights
|
| 39 |
-
|
| 40 |
-
JT-Math-8B-Thinking achieves its cutting-edge performance on complex mathematical challenges through a rigorous, multi-stage training methodology. Starting with the robust JT-Math-8B-Base model, our pipeline first implemented Supervised Fine-Tuning (SFT). This involved training on a high-quality, bilingual dataset of intricate math problems with 32,768-token context window. Subsequently, an advanced Reinforcement Learning (RL) phase, incorporating a multi-stage curriculum of progressively harder problems, further honed its reasoning abilities.
|
| 41 |
-
|
| 42 |
|
| 43 |
|
| 44 |
|
| 45 |
## Model Downloads
|
| 46 |
|
| 47 |
-
We release the following
|
| 48 |
|
| 49 |
-
| Model Name
|
| 50 |
-
| ------------------- |
|
| 51 |
-
| JT-Math-8B-Thinking | 32K
|
|
|
|
| 52 |
|
| 53 |
|
| 54 |
|
|
@@ -57,7 +52,7 @@ We release the following model to support a wide range of applications.
|
|
| 57 |
## Evaluation Results
|
| 58 |
|
| 59 |
JT-Math-8B-Thinking achieves competitive performance among open-source models in the ~8B class on mathematical reasoning benchmarks.
|
| 60 |
-
|
| 61 |
|
| 62 |
|
| 63 |
|
|
@@ -71,7 +66,7 @@ This example shows how to use the `JT-Math-8B-Thinking` model to solve math prob
|
|
| 71 |
```python
|
| 72 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 73 |
|
| 74 |
-
model_name = "
|
| 75 |
|
| 76 |
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
|
| 77 |
model = AutoModelForCausalLM.from_pretrained(
|
|
|
|
| 12 |
<a href="https://huggingface.co/JT-LM/JT-Math-8B-Thinking" target="_blank">
|
| 13 |
<img src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue">
|
| 14 |
</a>
|
| 15 |
+
<a href="https://www.modelscope.cn/models/JiuTian-AI/JT-Math-8B-Thinking" target="_blank">
|
| 16 |
+
<img src="https://img.shields.io/badge/%F0%9F%A4%96%20ModelScope-Models-blue">
|
| 17 |
</a>
|
| 18 |
</p>
|
| 19 |
|
|
|
|
| 28 |
|
| 29 |
|
| 30 |
|
| 31 |
+
## Model Details
|
| 32 |
|
| 33 |
+
The performance of **JT-Math-8B-Thinking** stems from a meticulous, multi-stage training approach aimed at tackling complex mathematical challenges with state-of-the-art accuracy. Building on the **JT-Math-8B-Base** model, its training pipeline involved **Supervised Fine-Tuning (SFT)** using a high-quality, bilingual dataset of intricate math problems. This SFT phase leveraged the model's native **32,768-token context window**, enabling it to comprehend lengthy premises, multi-step instructions, and problems with extensive background information right from the start. Following SFT, an advanced **Reinforcement Learning (RL)** phase further refined its reasoning capabilities. This RL process employed a multi-stage curriculum, gradually introducing problems of increasing difficulty, and was specifically engineered to boost the model's focus and accuracy across the entire 32K context window, ensuring the coherence and precision of even the longest reasoning chains.
|
| 34 |
|
| 35 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
|
| 38 |
|
| 39 |
## Model Downloads
|
| 40 |
|
| 41 |
+
We release the following models to support a wide range of applications.
|
| 42 |
|
| 43 |
+
| Model Name | Context Length | Hugging Face Link | ModelScope Link | Notes |
|
| 44 |
+
| ------------------- | -------------- | ---------------------------------------------------------- | ---------------------------------------------------------- | ---------------------------------------------------------- |
|
| 45 |
+
| JT-Math-8B-Thinking | 32K | [Link](https://huggingface.co/JT-LM/JT-Math-8B-Thinking) | [Link](https://www.modelscope.cn/models/JiuTian-AI/JT-Math-8B-Thinking) | The premier model for complex, long-context reasoning. |
|
| 46 |
+
------
|
| 47 |
|
| 48 |
|
| 49 |
|
|
|
|
| 52 |
## Evaluation Results
|
| 53 |
|
| 54 |
JT-Math-8B-Thinking achieves competitive performance among open-source models in the ~8B class on mathematical reasoning benchmarks.
|
| 55 |
+

|
| 56 |
|
| 57 |
|
| 58 |
|
|
|
|
| 66 |
```python
|
| 67 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 68 |
|
| 69 |
+
model_name = "JT-LM/JT-Math-8B-Thinking"
|
| 70 |
|
| 71 |
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
|
| 72 |
model = AutoModelForCausalLM.from_pretrained(
|