Datasets and models from "The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gains" (https://arxiv.org/abs/2507.06187).