Improve model card: Add pipeline tag, library name, paper and code links

README.md

@@ -1,7 +1,84 @@
 ---
-license: mit
-datasets:
-- Open-Reasoner-Zero/orz_math_57k_collection
 base_model:
 - Qwen/Qwen2.5-7B
----
+datasets:
+- Open-Reasoner-Zero/orz_math_57k_collection
+license: mit
+pipeline_tag: text-generation
+library_name: transformers
+tags:
+- code-generation
+- tool-use
+- mathematical-reasoning
+- rlhf
+---
+
+# Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving
+
+This repository contains the **ZeroTIR** model, a large language model fine-tuned for mathematical problem solving through spontaneous Python code generation and execution. This model was introduced in the paper:
+
+📚 [**Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving**](https://huggingface.co/papers/2505.07773)
+
+## Model Description
+
+Large Language Models (LLMs) often struggle with mathematical reasoning tasks requiring precise, verifiable computation. While Reinforcement Learning (RL) from outcome-based rewards enhances text-based reasoning, understanding how agents autonomously learn to leverage external tools like code execution remains crucial. This work investigates RL from outcome-based rewards for Tool-Integrated Reasoning (ZeroTIR), training base LLMs to spontaneously generate and execute Python code for mathematical problems without supervised tool-use examples. The central contribution is demonstrating that as RL training progresses, key metrics scale predictably. Specifically, strong positive correlations are observed where increased training steps lead to increases in spontaneous code execution frequency, average response length, and critically, final task accuracy. This suggests a quantifiable relationship between computational effort invested in training and the emergence of effective, tool-augmented reasoning strategies. ZeroTIR significantly surpasses non-tool ZeroRL baselines on challenging math benchmarks.
+
+## Usage
+
+This model can be loaded and used with the `transformers` library for text generation tasks, specifically for mathematical problem solving by enabling code execution.
+
+```python
+import torch
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+# Assuming the model ID on the Hugging Face Hub is named as below:
+model_id = "Open-Reasoner-Zero/Agent-RL-Scaling-Law-ZeroTIR-Qwen2.5-7B" 
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    torch_dtype=torch.bfloat16, 
+    device_map="auto"
+)
+
+# Example for mathematical problem solving with spontaneous code execution
+# The model is trained to generate code blocks to solve problems.
+prompt = (
+    "A rectangle has a perimeter of 40 units. Its length is 3 times its width. "
+    "What is the area of the rectangle? Provide your reasoning and use Python code to verify your answer.\
+"
+    "```python\
+"
+)
+
+input_ids = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+outputs = model.generate(
+    **input_ids,
+    max_new_tokens=512,
+    do_sample=True,
+    temperature=0.7,
+    top_p=0.9,
+    eos_token_id=tokenizer.eos_token_id
+)
+generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(generated_text)
+```
+
+## Code
+
+The official code and further details regarding this work can be found on the GitHub repository:
+
+🔗 [**https://github.com/yyht/openrlhf_async_pipline**](https://github.com/yyht/openrlhf_async_pipline)
+
+## Citation
+
+If you use this model or the associated research, please cite the paper:
+
+```bibtex
+@article{,
+  title={Agent RL Scaling Law: Agent RL with Spontaneous Code Execution for Mathematical Problem Solving},
+  author={},
+  journal={arXiv preprint arXiv:2505.07773},
+  year={2025}
+}
+```