evidence-multi-rlm-4b-grpo-step100

GRPO-trained multi-paper evidence-extraction RLM. 100 GRPO steps on top of alphaXiv/evidence-multi-rlm-sft-4b, trained with SkyRL using an LLM-judge reward.

Gathered from the FSDP world_size=8 training checkpoint (final-judge/global_step_100) into a single HF directory. Loads exactly like the SFT checkpoints.

Eval

Multi-paper evidence validation set (32 prompts), greedy decoding, judged by openai/gpt-5.4-mini. Reward is (judge_precision + judge_recall) / 2.

Model	avg_score	pass_at_1
this checkpoint	0.52 +/- 0.01 (3 runs)	0.94-0.97
evidence-multi-rlm-sft-4b (no RL)	0.4172	0.9688

Not bitwise reproducible even at temperature 0: vLLM continuous batching is batch-composition dependent and the judge is not fully deterministic.

Behaviour: submits a well-formed answer on 31/32 prompts, all via FINAL_VAR. Uses the intended parent/child decomposition (select papers -> dispatch child agents per paper -> aggregate spans). Known failure modes: occasional repetition to the length cap (3/531 turns), and sometimes prompting children with a question rather than an extraction instruction, which yields answers instead of verbatim spans.

Usage

from transformers import AutoModelForImageTextToText, AutoTokenizer

model = AutoModelForImageTextToText.from_pretrained("alphaXiv/evidence-multi-rlm-4b-grpo-step100", dtype="bfloat16")
tok = AutoTokenizer.from_pretrained("alphaXiv/evidence-multi-rlm-4b-grpo-step100")

The vision tower is carried over from the base model and is unused by the text-only evidence task.