YAML Metadata Warning:empty or missing yaml metadata in repo card
Check out the documentation for more information.
nbpo-checkpoints
Backups of the training runs for Nash Bargaining Preference Optimization (NBPO). They exist so a run can resume after the compute container is recycled; treat them as run state, not as a release.
Panels: helpsteer2 and hs2gemma are the four-judge HelpSteer2 panel with a Zephyr-7B-SFT and a
Gemma-2-9B-it reference respectively; saferlhf is the two-judge safety panel. A directory named
rmpool_mb<tag><r> is round r of the chain <tag>, whose anchor is round r-1 of the same chain.
Restore with resume_from_hf.sh from the project repository.
Contents
Each chain's most recent completed round, refreshed as the run advances. Older rounds are intentionally not kept: resuming a chain only needs its last policy.
Licenses
Models under hs2gemma/ are derived from google/gemma-2-9b-it and are distributed under the
Gemma Terms of Use; the Gemma Prohibited Use Policy applies to them
and to any further derivative. Models under helpsteer2/ and saferlhf/ are derived from
alignment-handbook/zephyr-7b-sft-full (Apache-2.0, itself derived from Mistral-7B).