| # Executive summary |
|
|
|
|
| **Fresh CPU replay (2026-08-03).** Seven public verification programs were |
| run again in this environment, including the exact 6,350-cell quadratic |
| lower-bound certificate and wider FedDPO scope sweep. See the |
| [run manifest](../../fresh_rerun_20260803/RUN_MANIFEST.json) and |
| [provenance](../../PROVENANCE.md). |
|
|
| Reproduction of *Distributed Direct Preference Optimization for LLM Alignment |
| under Heterogeneity* using the paper's DistilGPT-2 (82M) and Stanford Human |
| Preferences setting: five domain-disjoint clients, a frozen reference policy, |
| true token-level DPO loss, batch size 4, and the paper's FedDPO/DecDPO update |
| rules. |
|
|
| ## Real-model headline results |
|
|
| | registered mechanism | executed setting | decisive numerical result | |
| | --- | --- | --- | |
| | FedDPO local steps | `S=3,R=10,E=1,3,6` | loss `0.682574 → 0.664476 → 0.644070`; gradient-norm² `398.754 → 378.991 → 360.486` | |
| | FedDPO participation | `S=1,2,3,5`, `R=40,E=2` | parameter variance `5.0228e−06 → 1.2537e−09`; `4006x` collapse to full participation | |
| | async FedDPO staleness | `q_max=0,2,5`, `R=40,E=2` | q=2 mean loss `0.632112` versus synchronous `0.631802` | |
| | DecDPO topology | complete/ring/star/path | consensus error `4.8330e−05 → 1.5837e−03`, ordered by `1/(1−ρ²)` | |
|
|
| ## Claim-level interpretation |
|
|
| Theorem 5.1 is supported by its exact 432-cell term audit and by the real |
| FedDPO local-step and participation controls. Theorem 5.4 is supported by an |
| exact rational audit in which the staleness term is zero at q=0 and exactly |
| linear in q, plus the real q=2 degradation. Theorem 5.5 is supported by the |
| exact quadratic lower-bound enumeration on every registered population, |
| participation, local-step, and heterogeneity cell. Theorem 6.1 follows from a |
| direct substitution of `η=c/√R` into its displayed inequality and the real |
| topology ordering on full 82M parameter vectors. |
|
|
