# Real-model setup — DistilGPT-2 (82M) on Stanford Human Preferences

## Model and objective

The policy is `distilgpt2` (**82M parameters**) and the reference is a frozen
copy of the same checkpoint. The true DPO objective uses `β=0.1`:

```text
loss = −log sigmoid(β[(log π(yw|x)−log πref(yw|x))
                    −(log π(yl|x)−log πref(yl|x))]).
```

Log probabilities are summed over completion tokens only; prompt tokens are
masked. Every local update uses batch size 4 and learning rate `2e−5` unless a
decaying schedule is stated on the claim page.

## Data and clients

Pairs are streamed from `stanfordnlp/SHP`, ordered by the recorded preference
label, tokenized jointly to 128 tokens, and split by domain into five
non-IID clients: `askacademia_train`, `askanthropology_train`,
`askbaking_train`, `askcarguys_train`, and `askculinary_train`, 90 pairs each.
The domain split is the source of the client heterogeneity used in the bounds.

## Independent controls

| stream | seed | isolated quantity |
| --- | ---: | --- |
| client batch order | `777+i` | fixed data order within client i |
| client sampling | `10000+s` | participation variance across runs |
| stale-delay draws | `50000+s` | asynchronous delay variation |

With full participation, all client batch streams are fixed and the three
seeds return final loss `0.6318` to four decimals. The claim pages report the
exact number of clients, pairs, rounds, local steps, denominators, and seed
counts for each experiment.