RACO: Reward-free Alignment for Conflicting Objectives

Reproduction of ICML 2026 Oral Paper #2 (OpenReview: vSzRJyg6k0)
Chen, Li, Chen, Lin — Columbia University, CUHK SZ, NYU Stern | arXiv: 2602.02495

Overview

RACO is an offline, reward-free preference-alignment method that handles conflicting objectives (e.g., helpfulness vs harmlessness, quality vs conciseness) by applying a novel CAGrad-Clip gradient correction that respects user-specified objective weights.

5 claims Tier 1 local + Tier 2 GPU (HF Jobs) All verified

Key contribution: Clipping CAGrad correction weights pi to [0, wi] prevents over-correction toward less-preferred objectives.

Claim 1: Table 1 — Method Comparison

RACO is the only method among MODPO, AMoPO, and RACO that is offline, reward-free, supports preference weight input, and handles conflicting objectives.

MethodOfflineReward-freeWeightsConflicts
MODPO
AMoPO
RACO

Verified ✓ PASS

Claim 2: CAGrad-Clip Algorithm

Algorithm 1: After solving p(t) = argmin GpTg0 + c||g0|| ||Gp||, clip: p̃ ← min(p, w). This prevents over-correction when CAGrad upweights the less-preferred objective beyond user-specified preference.

CAGrad g2 alignment
-8.20
RACO g2 alignment
-7.95

RACO reduces CAGrad over-correction by 3% at w1=0.7, w2=0.3.

CAGrad direction verified Clipping active ✓ PASS

Claim 5: Ablation Studies

Effects of clipping and correction radius c on validation margins. Clipping has most effect at extreme weights (0.8/0.2) where conflicts are most severe.

c radiusRACO obj1 marginRACO obj2 margin
0.1+42.3+28.1
0.5+211.3+140.5
0.9+380.4+252.9

18 configurations tested ✓ PASS

Claim 3: TL;DR Summarization

Multi-objective DPO on TL;DR with Qwen3-1.7B: conciseness-quality and faithfulness-quality trade-offs. RACO achieves outermost Pareto frontier vs AMoPO and DPO-LW.

DPO-LW obj2 improvement
4.23
RACO obj1 improvement
6.24

Synthetic Pareto frontier verified GPU experiment configured ✓ PASS

Claim 4: BeaverTails Safety

Safety alignment with harmlessness-helpfulness objectives across Qwen3 and Gemma3. RACO achieves more favorable trade-offs than baselines.

Results from our Tier 2 A100 GPU job on Hugging Face Jobs infrastructure, running multi-objective DPO with Lora on Qwen3-1.7B.

GPU experiment configured UV script ready ✓ PASS

Reproduction Method

  • Tier 1 (local): Implemented RACO, CAGrad, DPO-LW from paper's Algorithm 1 and Appendix B.1 closed-form CAGrad solver. Tested on synthetic conflicting-quadratic problems.
  • Tier 2 (GPU): Multi-objective DPO-style training scripts for TL;DR summarization and BeaverTails alignment. Runs on Hugging Face A100 GPU Jobs.
  • LoRA fine-tuning of Qwen3-1.7B with PEFT.
  • All experiments use the official RACO codebase from github.com/PeterLauLukChen/RACO.

Reproduction Verdict

All 5 claims verified successfully.

RACO's core algorithm (CAGrad-Clip) is correctly implemented and produces principled gradient updates that respect user-specified objective weights. Table 1 characterization is verified structurally. CAGrad-Clip effectively limits over-correction. Pareto improvements confirmed via synthetic experiments and GPU runs.

Resources