Generating professional scholarly content, such as peer reviews and rebuttals, requires an intricate synergy between domain reasoning and factual grounding. This work presents a comprehensive framework for the development and evaluation of specialized scholarly agents, InternReviewer and InternAdvocate. We first establish a large-scale, high-quality scholarly dataset and integrate a high-efficiency arXiv retrieval tool to enable active evidence gathering. To optimize these agents, we implement an agentic Reinforcement Learning (RL) paradigm driven by a unified objective metric and reward system. This system avoids the biases of subjective model-based judging by employing multi-dimensional criteria, including reference-anchored semantic alignment, structural compliance, and a strict verification mechanism that cross-checks citations against real-time interaction logs to eliminate hallucinations. Experimental results demonstrate that agents trained within this closed-loop framework exhibit significant improvements in reasoning depth and citation accuracy.

Data Collection and Curation

Leveraging the official OpenReview API (https://github.com/openreview/openreview-py), we systematically curate a large-scale collection of peer review and rebuttal trajectories for manuscripts published through 2025. Our collection encompasses flagship computer science conferences, specifically ICLR (2013–2025), NeurIPS (2021–2024), and ICML (2025), resulting in a total of 184,857 unique review-rebuttal pairs alongside their original manuscript PDFs. See figure below for the relevant review and rebuttal data collection and curation pipeline.

Agentic Reinforcement Learning Framework

The following figure presents the Agentic Reinforcement Learning framework behind InternReviewer and InternAdvocate. The framework enables agents to interact with a time-aware scholarly retrieval environment, perform evidence-grounded reasoning, and optimize their behaviors through a decomposed objective reward covering semantic quality, structural compliance, tool usage, and citation verification. This closed-loop RL paradigm trains specialized academic agents capable of generating more rigorous, grounded, and reliable peer reviews and rebuttals.