MAHALO: Process Reward Model

This repository contains a Process Reward Model (PRM) introduced in the paper Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards.

MAHALO (Multi-Action-Head ALignment with PRM-guided DecOding) is a unified framework that standardizes PRM training across verifiable and non-verifiable settings for step-level supervision. It performs vectorized multi-objective alignment and enables controllable inference through objective-specific weighting and PRM-guided decoding.

Resources

Paper: Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards
Code: GitHub - pearls-lab/multiobj-align

Model Description

This model is a process-level reward model based on the Qwen2 architecture. It is designed to provide dense, step-level reward signals to align large language models across multiple objectives, including mathematical reasoning, human values, and interactive tutoring.

Citation

@article{shen2025simultaneous,
  title={Simultaneous Multi-objective Alignment Across Verifiable and Non-verifiable Rewards},
  author={Shen, Yiran and Xia, Yu and Chang, Jonathan and Ammanabrolu, Prithviraj},
  journal={arXiv preprint arXiv:2510.01167},
  year={2025},
  url={https://arxiv.org/abs/2510.01167}
}