Papers
arxiv:2610.13157

SOAR: Source-Aware Reinforcement Learning with Precise Credit Assignment for Search Agents

Published on Oct 9
Authors:
,
,
,
,
,
,
,
,
,
,
,
,

Abstract

SOAR improves reinforcement learning search agents by explicitly separating external and internal knowledge streams to reduce fabricated evidence and enhance reasoning reliability.

Reinforcement learning (RL) for search agents often suffers from evidence fabrication, a failure mode where agents bypass external information (E_{ext}) and fabricate an incorrect answer. We argue that this fabrication arises from an over-reliance on unreliable internal knowledge (E_{int}) as shortcuts due to the inherent opacity of reasoning trajectories. To address this, we propose SOAR (SOurce-Aware Reinforcement learning), a paradigm that transforms monolithic reasoning into source-explicit trajectories. SOAR (i) disentangles E_{ext} and E_{int} into parallel, observable streams via a Source-Aware Workflow, and (ii) incorporates Source-Consistent Reward to guide the policy evolution. By making information-source attribution explicit, SOAR enables precise credit assignment between different knowledge streams. Evaluations show that SOAR outperforms baselines on 3B and 7B backbones by 7.3% and 5.4%, while reducing evidence fabrication by 74.8%. These confirm that SOAR significantly improves performance on knowledge-intensive benchmarks while effectively mitigating evidence fabrication, fostering more faithful and grounded reasoning with enhanced robustness.

Community

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.13157
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2610.13157 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2610.13157 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.13157 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.