README.md

metadata

pipeline_tag: text-generation
library_name: transformers
license: apache-2.0

Learning Adaptive Parallel Reasoning
with Language Models

Jiayi Pan^*, Xiuyu Li^*, Long Lian^*, Charlie Victor Snell, Yifei Zhou,
Adam Yala, Trevor Darrell, Kurt Keutzer, Alane Suhr

UC Berkeley and UCSF    ^* Equal Contribution

📃 Paper • 💻 Code

TL;DR: We present Adaptive Parallel Reasoning (APR), a novel framework that enables language models to learn to orchestrate both serialized and parallel computations. APR trains language models to use spawn() and join() operations through end-to-end supervised training and reinforcement learning, allowing models to dynamically orchestrate their own computational workflows. APR efficiently distributes compute, reduces latency, overcomes context window limits, and achieves state‑of‑the‑art performance on complex reasoning tasks (e.g., 83.4% vs. 60.0% accuracy at 4K context on Countdown).

Citation

If you find this work useful in your research, please consider citing:

@article{pan2025learning,
  title   = {Learning Adaptive Parallel Reasoning with Language Models},
  author  = {Jiayi Pan and Xiuyu Li and Long Lian and Charlie Snell and Yifei Zhou and Adam Yala and Trevor Darrell and Kurt Keutzer and Alane Suhr},
  year    = {2025},
  journal = {arXiv preprint arXiv: 2504.15466}
}