Improve language tag

README.md

@@ -1,77 +1,91 @@
----
-base_model:
-- Qwen/Qwen2.5-7B-Instruct
-library_name: transformers
-license: mit
-pipeline_tag: text-generation
-tags:
-- reasoning
-- Zero-RL
----
-
-# 📖Introduction
-
-![Github](https://img.shields.io/badge/LUFFY-000000?style=for-the-badge&logo=github&logoColor=000&logoColor=white)
-
-LUFFY is a reinforcement learning framework that bridges the gap between zero-RL and imitation learning by incorporating off-policy reasoning traces into the training process. Built upon GRPO, LUFFY combines on-policy rollouts with off-policy demonstrations during advantage estimation and introduces **policy shaping** via regularized importance sampling to emphasize low-probability yet crucial actions.
-
-### Key Highlights:
-- **Off-Policy Guidance:** Seamlessly integrates external reasoning traces to bootstrap learning from stronger models.
-- **Dynamic Balance:** Learns when to imitate and when to explore, adapting over the course of training.
-- **Policy Shaping:** Emphasizes important actions often ignored in standard policy gradients, enabling better generalization.
-
----
-
-## Inference
-
-Here’s an example of using LUFFY for inference:
-
-
-```python
-from transformers import AutoTokenizer
-from vllm import LLM, SamplingParams
-
-model_path="Elliott/LUFFY-Qwen-Math-7B-Zero"
-
-question = "which number is larger? 9.11 or 9.9?"
-
-tokenizer = AutoTokenizer.from_pretrained(model_path)
-messages = [{"role": "user", "content": question}]
-chat = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
-
-llm = LLM(model=model_path)
-params = SamplingParams(temperature=0.6, max_tokens=8192)
-outputs = llm.generate([chat], params)
-print(outputs[0].outputs[0].text)
-```
-
----
-
-# 📃Evaluation
-
-| **Model**                          | **AIME 2024** | **AIME 2025** | **AMC** | **MATH-500** | **Minerva** | **Olympiad** | **Avg.** |
-|-----------------------------------|-------------|-------------|---------|---------------|-------------|---------------|----------|
-| Qwen2.5-7B-Instruct           | 11.9 | 7.6 | 44.1 | 74.6 | 30.5 | 39.7 | 34.7 |
-| **LUFFY-Qwen-Instruct-7B**             | **16.6** | **15.7** | **52.2**  | **81.4**  | **36.8**  | **48.7**  | **41.9** |
-
----
-
-# 🌻Acknowledgement
-
-LUFFY builds upon [veRL](https://github.com/volcengine/verl) and [deepscaler](https://github.com/agentica-project/rllm), and utilizes [vLLM](https://github.com/vllm-project/vllm) for inference. We utilize [Math-Verify](https://github.com/huggingface/Math-Verify) for math reasoning evaluation. We thank the open-source community for datasets and backbones, including [NuminaMath](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT), [OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k), [Qwen2.5-Math](https://github.com/QwenLM/Qwen2.5-Math), and [DeepSeek-R1](https://github.com/deepseek-ai/deepseek-r1) model. 
-
-Code: https://github.com/ElliottYan/LUFFY
-
-# Citation
-If you find our model, data, or evaluation code useful, please kindly cite our paper:
-```bib
-@misc{luffy,
-      title={Learning to Reason under Off-Policy Guidance}, 
-      author={Jianhao Yan and Yafu Li and Zican Hu and Zhi Wang and Ganqu Cui and Xiaoye Qu and Yu Cheng and Yue Zhang},
-      year={2025},
-      eprint={2504.14945},
-      archivePrefix={arXiv},
-      primaryClass={cs.LG},
-      url={https://arxiv.org/abs/2504.14945}, 
-}
+---
+base_model:
+- Qwen/Qwen2.5-7B-Instruct
+library_name: transformers
+license: mit
+pipeline_tag: text-generation
+tags:
+- reasoning
+- Zero-RL
+language:
+- zho
+- eng
+- fra
+- spa
+- por
+- deu
+- ita
+- rus
+- jpn
+- kor
+- vie
+- tha
+- ara
+---
+
+# 📖Introduction
+
+![Github](https://img.shields.io/badge/LUFFY-000000?style=for-the-badge&logo=github&logoColor=000&logoColor=white)
+
+LUFFY is a reinforcement learning framework that bridges the gap between zero-RL and imitation learning by incorporating off-policy reasoning traces into the training process. Built upon GRPO, LUFFY combines on-policy rollouts with off-policy demonstrations during advantage estimation and introduces **policy shaping** via regularized importance sampling to emphasize low-probability yet crucial actions.
+
+### Key Highlights:
+- **Off-Policy Guidance:** Seamlessly integrates external reasoning traces to bootstrap learning from stronger models.
+- **Dynamic Balance:** Learns when to imitate and when to explore, adapting over the course of training.
+- **Policy Shaping:** Emphasizes important actions often ignored in standard policy gradients, enabling better generalization.
+
+---
+
+## Inference
+
+Here’s an example of using LUFFY for inference:
+
+
+```python
+from transformers import AutoTokenizer
+from vllm import LLM, SamplingParams
+
+model_path="Elliott/LUFFY-Qwen-Math-7B-Zero"
+
+question = "which number is larger? 9.11 or 9.9?"
+
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+messages = [{"role": "user", "content": question}]
+chat = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+
+llm = LLM(model=model_path)
+params = SamplingParams(temperature=0.6, max_tokens=8192)
+outputs = llm.generate([chat], params)
+print(outputs[0].outputs[0].text)
+```
+
+---
+
+# 📃Evaluation
+
+| **Model**                          | **AIME 2024** | **AIME 2025** | **AMC** | **MATH-500** | **Minerva** | **Olympiad** | **Avg.** |
+|-----------------------------------|-------------|-------------|---------|---------------|-------------|---------------|----------|
+| Qwen2.5-7B-Instruct           | 11.9 | 7.6 | 44.1 | 74.6 | 30.5 | 39.7 | 34.7 |
+| **LUFFY-Qwen-Instruct-7B**             | **16.6** | **15.7** | **52.2**  | **81.4**  | **36.8**  | **48.7**  | **41.9** |
+
+---
+
+# 🌻Acknowledgement
+
+LUFFY builds upon [veRL](https://github.com/volcengine/verl) and [deepscaler](https://github.com/agentica-project/rllm), and utilizes [vLLM](https://github.com/vllm-project/vllm) for inference. We utilize [Math-Verify](https://github.com/huggingface/Math-Verify) for math reasoning evaluation. We thank the open-source community for datasets and backbones, including [NuminaMath](https://huggingface.co/datasets/AI-MO/NuminaMath-CoT), [OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k), [Qwen2.5-Math](https://github.com/QwenLM/Qwen2.5-Math), and [DeepSeek-R1](https://github.com/deepseek-ai/deepseek-r1) model. 
+
+Code: https://github.com/ElliottYan/LUFFY
+
+# Citation
+If you find our model, data, or evaluation code useful, please kindly cite our paper:
+```bib
+@misc{luffy,
+      title={Learning to Reason under Off-Policy Guidance}, 
+      author={Jianhao Yan and Yafu Li and Zican Hu and Zhi Wang and Ganqu Cui and Xiaoye Qu and Yu Cheng and Yue Zhang},
+      year={2025},
+      eprint={2504.14945},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG},
+      url={https://arxiv.org/abs/2504.14945}, 
+}
 ```