Executive summary
Fresh CPU replay (2026-08-03). Seven public verification programs were run again in this environment, including the exact 6,350-cell quadratic lower-bound certificate and wider FedDPO scope sweep. See the run manifest and provenance.
Reproduction of Distributed Direct Preference Optimization for LLM Alignment under Heterogeneity using the paper's DistilGPT-2 (82M) and Stanford Human Preferences setting: five domain-disjoint clients, a frozen reference policy, true token-level DPO loss, batch size 4, and the paper's FedDPO/DecDPO update rules.
Real-model headline results
| registered mechanism | executed setting | decisive numerical result |
|---|---|---|
| FedDPO local steps | S=3,R=10,E=1,3,6 |
loss 0.682574 → 0.664476 → 0.644070; gradient-norm² 398.754 → 378.991 → 360.486 |
| FedDPO participation | S=1,2,3,5, R=40,E=2 |
parameter variance 5.0228e−06 → 1.2537e−09; 4006x collapse to full participation |
| async FedDPO staleness | q_max=0,2,5, R=40,E=2 |
q=2 mean loss 0.632112 versus synchronous 0.631802 |
| DecDPO topology | complete/ring/star/path | consensus error 4.8330e−05 → 1.5837e−03, ordered by 1/(1−ρ²) |
Claim-level interpretation
Theorem 5.1 is supported by its exact 432-cell term audit and by the real
FedDPO local-step and participation controls. Theorem 5.4 is supported by an
exact rational audit in which the staleness term is zero at q=0 and exactly
linear in q, plus the real q=2 degradation. Theorem 5.5 is supported by the
exact quadratic lower-bound enumeration on every registered population,
participation, local-step, and heterogeneity cell. Theorem 6.1 follows from a
direct substitution of η=c/√R into its displayed inequality and the real
topology ordering on full 82M parameter vectors.