Improve language tag
#1
by lbourdois - opened
README.md
CHANGED
|
@@ -1,129 +1,143 @@
|
|
| 1 |
-
---
|
| 2 |
-
library_name: transformers
|
| 3 |
-
base_model:
|
| 4 |
-
- Qwen/Qwen2.5-32B-Instruct
|
| 5 |
-
- infly/inf-o1-pi0
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
##
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
##
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
|
| 55 |
-
|
|
| 56 |
-
|
|
| 57 |
-
|
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
|
| 61 |
-
|
|
| 62 |
-
| Qwen2.5-32B-
|
| 63 |
-
|
|
| 64 |
-
| INF-o1-pi0 |
|
| 65 |
-
##
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
-
|
| 80 |
-
|
| 81 |
-
|
| 82 |
-
|
| 83 |
-
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
|
| 87 |
-
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
|
| 94 |
-
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
]
|
| 99 |
-
|
| 100 |
-
|
| 101 |
-
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
| 111 |
-
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
| 115 |
-
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
|
| 119 |
-
##
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
| 123 |
-
|
| 124 |
-
|
| 125 |
-
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 129 |
```
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
base_model:
|
| 4 |
+
- Qwen/Qwen2.5-32B-Instruct
|
| 5 |
+
- infly/inf-o1-pi0
|
| 6 |
+
language:
|
| 7 |
+
- zho
|
| 8 |
+
- eng
|
| 9 |
+
- fra
|
| 10 |
+
- spa
|
| 11 |
+
- por
|
| 12 |
+
- deu
|
| 13 |
+
- ita
|
| 14 |
+
- rus
|
| 15 |
+
- jpn
|
| 16 |
+
- kor
|
| 17 |
+
- vie
|
| 18 |
+
- tha
|
| 19 |
+
- ara
|
| 20 |
+
---
|
| 21 |
+
|
| 22 |
+
These are the original weights uploaded by infly, just swapped out the tokenizer config files for QwQ's version. Works better.
|
| 23 |
+
|
| 24 |
+
---
|
| 25 |
+
<div align="center">
|
| 26 |
+
<img src="INF.jpg" width="300"/>
|
| 27 |
+
|
| 28 |
+
🤗 <a href="https://huggingface.co/infly" target="_blank">Hugging Face</a>
|
| 29 |
+
<br>
|
| 30 |
+
<a href="https://inftech-pi-zero.github.io/" target="_blank">Github</a>
|
| 31 |
+
<br>
|
| 32 |
+
<br>
|
| 33 |
+
<br>
|
| 34 |
+
</div>
|
| 35 |
+
|
| 36 |
+
<div align="center">
|
| 37 |
+
<h1>INF-o1-pi0: Initiating the Journey to the Infinity of LLM Reasoning</h1>
|
| 38 |
+
<p>INF AI specializes in foundational large language model technology and applications. We develop trustworthy vertical-domain models and AI-native solutions tailored to industry needs. Our team of expert AI scientists and industry leaders focuses on practical "gray-box" technologies, unlocking the productivity of large language models to drive innovation across sectors. Our mission in the INF-o1 project is to enhance the reasoning capabilities of LLMs across various industrial domains and ensure a trustworthy reasoning process to serve industry needs.</p>
|
| 39 |
+
<p>INFLY TECH (Shanghai) Co., Ltd.</p>
|
| 40 |
+
<p>2024.12.31</p>
|
| 41 |
+
</div>
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
## Overview
|
| 46 |
+
We are pleased to share the initial checkpoint of our reasoning foundation large language model as an open-source resource. This checkpoint is intended to help evaluate our team's data production pipeline across various domains, including mathematics, programming, logic, safety, and others. Its goal is to provide a solid starting point for developing a robust policy for the subsequent reinforcement learning process.
|
| 47 |
+
|
| 48 |
+
We are hopeful that applying our reinforcement learning algorithms, supported by our carefully designed infrastructure, will lead to meaningful improvements in the model’s reasoning capabilities across various domains. At the heart of the project is our data production pipeline, which we believe plays a crucial role in enabling general reasoning capabilities. We also believe that the reasoning capability induced by the data production pipline can address a range of real-world industrial scenarios with increasing precision and reliability.
|
| 49 |
+
|
| 50 |
+
Based on our observations during the production of pi0, we have identified quality and diversity as critical factors for fostering high-quality, long Chain-of-Thought (CoT) reasoning capabilities. This insight aligns closely with conclusions drawn from the general alignment process of large language models. By meticulously designing self-verification and backtracking mechanisms to ensure process correctness in data generation, we have developed datasets that effectively induce robust long-context reasoning across diverse domains. This approach demonstrates superior performance compared to state-of-the-art o1-lile models with similar objectives, highlighting the potential of our data production pipline in advancing reasoning capabilities.
|
| 51 |
+
## Experiments
|
| 52 |
+
### Math Benchmarks
|
| 53 |
+
|
| 54 |
+
| Model | College Math | AMC23 | MATH | Olympiad Bench | GaoKao 2023 En | AIME24 |
|
| 55 |
+
| ---------------------- | ------------ | ----- | ----- | --------------- | -------------- | ------ |
|
| 56 |
+
| Qwen2.5-32B-Instruct | 45.71 | 72.5 | 82.82 | 46.81 | 68.83 | 23.33 |
|
| 57 |
+
| Qwen2.5-32B-QwQ | 43.33 | 72.5 | 88.54 | 55.56 | 78.70 | 40.00 |
|
| 58 |
+
| INF-o1-pi0 | 47.27 | 85.0 | 88.60 | 56.00 | 77.14 | 40.00 |
|
| 59 |
+
### Logical Benchmark
|
| 60 |
+
| Model | lsat |
|
| 61 |
+
| ----------------- | :---: |
|
| 62 |
+
| Qwen2.5-32B-Instruct | 33.7
|
| 63 |
+
| Qwen2.5-32B-QwQ | 67.0 |
|
| 64 |
+
| INF-o1-pi0 | 71.8 |
|
| 65 |
+
### Safety Benchmarks
|
| 66 |
+
| Model | AIR-BENCH 2024 | AIR-BENCH 2024(CRF) |
|
| 67 |
+
| ----------------- | :---: | :---: |
|
| 68 |
+
| Qwen2.5-32B-Instruct | 54.29 | 53.83 |
|
| 69 |
+
| Qwen2.5-32B-QwQ | 52.61 | 53.42 |
|
| 70 |
+
| o1-preview | 73.25 | 70.72 |
|
| 71 |
+
| INF-o1-pi0 | 77.25 | 74.49 |
|
| 72 |
+
### SQL Benchmarks
|
| 73 |
+
| Model | bird | spider |
|
| 74 |
+
| ----------------- | :---: | :---: |
|
| 75 |
+
| Qwen2.5-32B-Instruct | 50.2 | 77.8 |
|
| 76 |
+
| Qwen2.5-32B-QwQ | 43.7 | 69.9 |
|
| 77 |
+
| o1-preview | 48.9 | 70.6 |
|
| 78 |
+
| INF-o1-pi0 | 55.3 | 79.7 |
|
| 79 |
+
## Quick Start
|
| 80 |
+
We provide an example usage of the inf-o1-pi0 below.
|
| 81 |
+
```python
|
| 82 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 83 |
+
|
| 84 |
+
model_name = "infly/inf-o1-pi0"
|
| 85 |
+
|
| 86 |
+
model = AutoModelForCausalLM.from_pretrained(
|
| 87 |
+
model_name,
|
| 88 |
+
torch_dtype="auto",
|
| 89 |
+
device_map="auto"
|
| 90 |
+
)
|
| 91 |
+
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
| 92 |
+
|
| 93 |
+
prompt = "Give me a short introduction to large language model."
|
| 94 |
+
|
| 95 |
+
messages = [
|
| 96 |
+
{"role": "system", "content": "You are an advanced AI language model specializing in solving math and programming problems step by step. Carefully analyze each part of the problem, verify the accuracy of your reasoning with relevant facts and data, and provide clear, logical solutions. Reflect on and review your approach throughout the problem-solving process to ensure precision and thoroughness. Always think through the problem step by step and provide your answers accordingly."},
|
| 97 |
+
{"role": "user", "content": prompt}
|
| 98 |
+
]
|
| 99 |
+
text = tokenizer.apply_chat_template(
|
| 100 |
+
messages,
|
| 101 |
+
tokenize=False,
|
| 102 |
+
add_generation_prompt=True
|
| 103 |
+
)
|
| 104 |
+
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
|
| 105 |
+
|
| 106 |
+
generated_ids = model.generate(
|
| 107 |
+
**model_inputs,
|
| 108 |
+
max_new_tokens=512
|
| 109 |
+
)
|
| 110 |
+
generated_ids = [
|
| 111 |
+
output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
|
| 112 |
+
]
|
| 113 |
+
|
| 114 |
+
response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
| 115 |
+
print(response)
|
| 116 |
+
|
| 117 |
+
|
| 118 |
+
```
|
| 119 |
+
## Future Plan
|
| 120 |
+
Our pi0 serves as the foundation for ensuring that our data generation pipeline effectively leverages the long reasoning capabilities of large language models. Looking ahead, we plan to use pi0 as the initial policy checkpoint for reinforcement learning training. Through this process, we aim to significantly enhance the generalization of reasoning capabilities, particularly for tasks in the financial and medical domains, which are critical for both academic research and industrial applications.
|
| 121 |
+
## Contributor
|
| 122 |
+
### Supervisors
|
| 123 |
+
Wei Chu • Yinghui Xu • Yuan Qi
|
| 124 |
+
### INF-o1 team
|
| 125 |
+
**Listed in Alphabetical Order**
|
| 126 |
+
|
| 127 |
+
Chao Qu - Team Leader • Chao Wang - Infrastructure • Cheng Peng - Data Pipeline (Logical) • Dakuan Lu - Data Pipeline (Science) • Haozhe Wang - Data Pipeline (Math) & RL • Hongqing Hu - Infrastructure • Jianming Feng - Data Pipeline (Safety) • Jiaran Hao - Data Pipeline (SQL) & Infrastructure • Kelang Tian - Infrastructure • Minghao Yang - Data Pipeline (Math) • Quanbin Wang - Data Pipeline (Safety) • J.K. Liu - Data Pipeline (SQL) • Tianchu Yao - Data Pipeline & Alignment • Weidi Xu - Data Pipeline (Logical) • Xiaoyu Tan - Data Pipeline & Alignment • Yihan Songliu - Infrastructure
|
| 128 |
+
## License Agreement
|
| 129 |
+
infly-o1-pi0 support commercial applications under a permissive [License](https://huggingface.co/infly/inf-o1-pi0/blob/main/LICENSE).
|
| 130 |
+
## Contact
|
| 131 |
+
Chao Qu: quchao_tequila@inftech.ai
|
| 132 |
+
Xiaoyu Tan: yulin.txy@inftech.ai
|
| 133 |
+
## Cititation
|
| 134 |
+
If you find our work helpful, feel free to give us a cite.
|
| 135 |
+
```
|
| 136 |
+
@misc{inftech_pi_zero2024,
|
| 137 |
+
author = {INF-o1 Team},
|
| 138 |
+
title = {INF-o1 (\(\pi_0\)): Initiating the Journey to the Infinity of LLM Reasoning},
|
| 139 |
+
year = {2024},
|
| 140 |
+
url = {https://inftech-pi-zero.github.io/},
|
| 141 |
+
note = {Accessed: 2024-12-31}
|
| 142 |
+
}
|
| 143 |
```
|