# ActPRM
## Links
- [GitHub](https://github.com/sail-sg/ActivePRM)
- [ActivePRM Collection](https://huggingface.co/collections/sail/active-prm-67fccbc10c94c41a6d6b26d9)
## Introduction
We release our process reward model effectively trained with active learning based on `Qwen/Qwen2.5-Math-7B-Instruct`.
<!-- <p align="center">
<img src="https://raw.githubusercontent.com/sail-sg/ActivePRM/refs/heads/main/assets/figure1.png" width=70%/>
</p> -->
### Usage
We provide a code snippet to show the usage of our PRM.
```python
# modified from https://huggingface.co/Qwen/Qwen2.5-Math-PRM-7B/blob/main/README.md
from transformers import AutoModel, AutoTokenizer
def make_step_rewards(logits, token_masks):
res = []
for j in range(outputs.logits.size(1)):
logits = outputs.logits[:, j, token_masks[j]]
logits = logits.mean(dim=0)
res.append(logits.tolist())
return res
model_name = "sail/ActPRM"
device = "auto"
tokenizer = AutoTokenizer.from_pretrained(mo…
# ActPRM
## Links
- [GitHub](https://github.com/sail-sg/ActivePRM)
- [ActivePRM Collection](https://huggingface.co/collections/sail/active-prm-67fccbc10c94c41a6d6b26d9)
## Introduction
We release our process reward model effectively trained with active learning based on `Qwen/Qwen2.5-Math-7B-Instruct`.
<!-- <p align="center">
<img src="https://raw.githubusercontent.com/sail-sg/ActivePRM/refs/heads/main/assets/figure1.png" width=70%/>
</p> -->
### Usage
We provide a code snippet to show the usage of our PRM.
```python
# modified from https://huggingface.co/Qwen/Qwen2.5-Math-PRM-7B/blob/main/README.md
from transformers import AutoModel, AutoTokenizer
def make_step_rewards(logits, token_masks):
res = []
for j in range(outputs.logits.size(1)):
logits = outputs.logits[:, j, token_masks[j]]
logits = logits.mean(dim=0)
res.append(logits.tolist())
return res
model_name = "sail/ActPRM"
device = "auto"
tokenizer = AutoTokenizer.from_pretrained(mo…