Instructions to use PRIME-RL/Eurus-2-7B-PRIME with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use PRIME-RL/Eurus-2-7B-PRIME with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="PRIME-RL/Eurus-2-7B-PRIME")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("PRIME-RL/Eurus-2-7B-PRIME")
model = AutoModelForCausalLM.from_pretrained("PRIME-RL/Eurus-2-7B-PRIME")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use PRIME-RL/Eurus-2-7B-PRIME with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "PRIME-RL/Eurus-2-7B-PRIME"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PRIME-RL/Eurus-2-7B-PRIME",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/PRIME-RL/Eurus-2-7B-PRIME

SGLang

How to use PRIME-RL/Eurus-2-7B-PRIME with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "PRIME-RL/Eurus-2-7B-PRIME" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PRIME-RL/Eurus-2-7B-PRIME",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "PRIME-RL/Eurus-2-7B-PRIME" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "PRIME-RL/Eurus-2-7B-PRIME",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use PRIME-RL/Eurus-2-7B-PRIME with Docker Model Runner:
```
docker model run hf.co/PRIME-RL/Eurus-2-7B-PRIME
```

hanbin commited on Dec 31, 2024

Commit

a8dfc11

verified ·

1 Parent(s): f569aa3

Update README.md

Browse files

Files changed (1) hide show

README.md +13 -13

README.md CHANGED Viewed

@@ -7,15 +7,15 @@ license: apache-2.0
 - 📜 [Blog]()
 - 🤗 [PRIME Collection](https://huggingface.co/PRIME-RL)
-- 🤗 [RL Data]()
 ## Introduction
-Eurus-2-7B-PRIME is trained using **PRIME** (**P**rocess **R**einforcement through **IM**plicit r**E**ward) method, which effectively incorporates and updates reward models in reinforcement learning. It starts with [Eurus-2-7B-SFT](https://huggingface.co/PRIME-RL/Eurus-2-7B-SFT) and trains on [Eurus-2-RL-Data]().
 <img src="./figures/prm.gif" alt="prm" style="zoom: 33%;" />
-As shown in the animation above, in PRIME, the policy model and PRM are both initialized with the SFT model. For each RL iteration, the policy model first generates rollouts. Then, the [implicit PRM](https://arxiv.org/abs/2412.01981) and outcome verifier score the rollouts, and the implicit PRM get updated on the rollouts with outcome reward. Finally, the outcome reward \\(r_o\\) and process reward \\(r_p\\) are combined and used to update the policy model.
 The PRIME implementation pseudocode is as follows:
@@ -60,18 +60,18 @@ We apply tailored prompts for coding and math task:
 ## Evaluation
-Through PRIME, we successfully achieved substantial improvement on key reasoning benchmarks compared with the SFT model, leading to over **14.7%** improvement on average, over **20%** on AMC&AIME competitions.
 The final results are presented below:
-|               | **Eurus-2-7B-PRIME** | Epoch2-272step | **Eurus-2-7B-SFT** | **Qwen-2.5-Math-7B-Instruct** | **Llama-3.1-70B-Instruct** | **GPT-4o** |
-| ------------- | -------------------- | -------------- | ------------------ | ----------------------------- | -------------------------- | ---------- |
-| AIME 2024     | **23.3 (+20.0)**     | 26.7           | 3.3                | 13.3                          | 16.7                       | 9.3        |
-| MATH-500      | 77.2 (+12.1)         | 79.2           | 65.1               | **79.8**                      | 64.6                       | 76.4       |
-| AMC           | **55.4 (+25.3)**     | 57.8           | 30.1               | 50.6                          | 30.1                       | 45.8       |
-| Minerva Math  | **39.3 (+6.6)**      | 38.6           | 32.7               | 34.6                          | 35.3                       | 36.8       |
-| OlympiadBench | 39.3 (+9.5)          | 42.1           | 29.8               | 40.7                          | 31.9                       | **43.3**   |
-| Avg.          | **46.9 (+14.7)**     | 48.9           | 32.2               | 43.8                          | 36.4                       | 43.3       |
 ![image-20241230162026156](./figures/performance.jpg)
@@ -83,7 +83,7 @@ We achieved this with only 1/10 data and model resources compared with Qwen-Math
 | SFT Data   | **230K (open-source)**             | 2.5M (open-source and in-house) |
 | RM Data    | **0**                              | 618K (in-house)                 |
 | RM         | **Eurus-2-7B-SFT**                 | Qwen2.5-Math-RM (72B)           |
-| RL Data    | **80K queries  \\(\times\\)4 samples** | 66K queries \\(\times\\) 32 samples |

 - 📜 [Blog]()
 - 🤗 [PRIME Collection](https://huggingface.co/PRIME-RL)
+- 🤗 [RL Data](https://huggingface.co/datasets/PRIME-RL/Eurus-2-RL-Data)
 ## Introduction
+Eurus-2-7B-PRIME is trained using **PRIME** (**P**rocess **R**einforcement through **IM**plicit r**E**ward) method, an open-source solution for online reinforcement learning (RL) with process rewards, to advance reasoning abilities of language models beyond imitation or distillation. It starts with [Eurus-2-7B-SFT](https://huggingface.co/PRIME-RL/Eurus-2-7B-SFT) and trains on [Eurus-2-RL-Data](https://huggingface.co/datasets/PRIME-RL/Eurus-2-RL-Data).
 <img src="./figures/prm.gif" alt="prm" style="zoom: 33%;" />
+As shown in the animation above, in PRIME, the policy model and PRM are both initialized with the SFT model. For each RL iteration, the policy model first generates rollouts. Then, the [implicit PRM](https://arxiv.org/abs/2412.01981) and outcome verifier score the rollouts, and the implicit PRM gets updated on the rollouts with the outcome reward. Finally, the outcome reward \\(r_o\\) and process reward \\(r_p\\) are combined and used to update the policy model.
 The PRIME implementation pseudocode is as follows:
 ## Evaluation
+Through PRIME, we successfully achieve substantial improvements on key reasoning benchmarks over our SFT version of the model, leading to **16.7%** improvement on average, and over **20%** on AMC&AIME competitions. Our final model Eurus-2-7B-PRIME, based on Qwen-2.5-Math-7B-Base, surpassed its instruct version on 5 key reasoning benchmarks.
 The final results are presented below:
+|               | **Eurus-2-7B-PRIME** | **Eurus-2-7B-SFT** | **Qwen-2.5-Math-7B-Instruct** | **Llama-3.1-70B-Instruct** | **GPT-4o** |
+| ------------- | -------------------- | ------------------ | ----------------------------- | -------------------------- | ---------- |
+| AIME 2024     | **26.7 (+23.3)**     | 3.3                | 13.3                          | 16.7                       | 9.3        |
+| MATH-500      | 79.2 (+14.1)         | 65.1               | **79.8**                      | 64.6                       | 76.4       |
+| AMC           | **57.8 (+27.7)**     | 30.1               | 50.6                          | 30.1                       | 45.8       |
+| Minerva Math  | **38.6 (+5.9)**      | 32.7               | 34.6                          | 35.3                       | 36.8       |
+| OlympiadBench | 42.1 (+12.3)         | 29.8               | 40.7                          | 31.9                       | **43.3**   |
+| Avg.          | **48.9 (+ 16.7)**    | 32.2               | 43.8                          | 36.4                       | 43.3       |
 ![image-20241230162026156](./figures/performance.jpg)
 | SFT Data   | **230K (open-source)**             | 2.5M (open-source and in-house) |
 | RM Data    | **0**                              | 618K (in-house)                 |
 | RM         | **Eurus-2-7B-SFT**                 | Qwen2.5-Math-RM (72B)           |
+| RL Data    | **150K queries  \\(\times\\)4 samples** | 66K queries \\(\times\\) 32 samples |