proposal-rl Qwen2.5-7B — PPL-reward GRPO (V1)
Qwen2.5-7B-Instruct fine-tuned (SFT → GRPO RL) to generate research proposals from a reading list
(top-k references conditioning). RL reward = perplexity-based signal (PPL of the target paper's
abstract under the proposal) from the V1 proposal_rl pipeline; this checkpoint is the best
hyper-parameter variant of experiment exp17_top_k_refs_ppl_rl (lr 5e-6, KL 0.01; mean reward
0.3287 across the sweep).
- Base: Qwen/Qwen2.5-7B-Instruct (Apache-2.0)
- Data:
arxiv-research-proposals-v1(7,357 papers, temporal split) - Recipe: SFT then GRPO; full recipe, ablations (17 experiments), and eval protocol in the repo: https://github.com/XinghanLi66/proposal_rl
- Format: merged HF safetensors (single shard), ready for
AutoModelForCausalLM.
Intended use & limitations
Research-ideation experimentation. Proposals are on-topic and reference-grounded but the project's own benchmark analysis found pass@k-style implementation benchmarks only weakly reflect proposal quality (see repo reports). Not for factual claims; outputs are synthetic research ideas.
- Downloads last month
- -