Add metadata and improve model card

README.md

@@ -1,22 +1,33 @@
-# Official Repo of Reagent Agent Reasoning Reward Model (Agent-RRM).
-Paper: https://arxiv.org/abs/2601.22154
+---
+license: apache-2.0
+library_name: transformers
+pipeline_tag: reinforcement-learning
+---
 
-## Abstract:
-Agentic Reinforcement Learning (Agentic RL) has achieved notable success in enabling agents to perform complex reasoning and tool use.
-However, most methods still relies on sparse outcome-based reward for training.
-Such feedback fails to differentiate intermediate reasoning quality, leading to suboptimal training results.
-In this paper, we introduce \textbf{Agent Reasoning Reward Model (Agent-RRM)}, a multi-faceted reward model that produces structured feedback for agentic trajectories, including (1) an explicit reasoning trace , (2) a focused critique that provides refinement guidance by highlighting reasoning flaws, and (3) an overall score that evaluates process performance.
-Leveraging these signals, we systematically investigate three integration strategies: \textbf{Reagent-C} (text-augmented refinement), \textbf{Reagent-R} (reward-augmented guidance), and \textbf{Reagent-U} (unified feedback integration).
-Extensive evaluations across 12 diverse benchmarks demonstrate that Reagent-U yields substantial performance leaps, achieving 43.7\% on GAIA and 46.2\% on WebWalkerQA, validating the effectiveness of our reasoning reward model and training schemes.
+# Agent Reasoning Reward Model (Agent-RRM)
 
-## GitHub Repository
-The official codebase, including training and evaluation scripts for Reagent, can be found on the project's GitHub repository: https://github.com/kxfan2002/Reagent
+This repository contains the weights for **Agent-RRM**, introduced in the paper [Exploring Reasoning Reward Model for Agents](https://huggingface.co/papers/2601.22154).
 
-## Citation
+## Introduction
+Agent Reasoning Reward Model (Agent-RRM) is a multi-faceted reward model designed for Agentic Reinforcement Learning. Unlike traditional sparse outcome-based rewards, Agent-RRM provides structured feedback for agentic trajectories, including:
+1. **Explicit reasoning trace**: Step-by-step reasoning analysis.
+2. **Focused critique**: Refinement guidance highlighting reasoning flaws.
+3. **Overall score**: Evaluation of process performance.
 
-```bash
+These signals enable training strategies like **Reagent-U**, which has demonstrated significant performance leaps on benchmarks such as GAIA and WebWalkerQA.
 
-```
----
-license: apache-2.0
----
+## Resources
+- **Paper:** [Exploring Reasoning Reward Model for Agents](https://huggingface.co/papers/2601.22154)
+- **GitHub Repository:** [kxfan2002/Reagent](https://github.com/kxfan2002/Reagent)
+
+## Citation
+If you find this work helpful, please consider citing:
+
+```bibtex
+@article{fan2025exploring,
+  title={Exploring Reasoning Reward Model for Agents},
+  author={Kaixuan Fan and Kaituo Feng and Manyuan Zhang and Tianshuo Peng and Zhixun Li and Yilei Jiang and Shuang Chen and Peng Pei and Xunliang Cai and Xiangyu Yue},
+  journal={arXiv preprint arXiv:2601.22154},
+  year={2025}
+}
+```