Spico/Humback-Myx

🐋 Humback

The proposed Humback is a novel framework that can augment the instruction data for supervised fine-tuning with high quality.

This is a backward model $M_{yx}$ for Humback reproduction.

This model is trained on the seed data in a reversed order (generate instruction given response).

The seed data is a sampled dataset from oasst1.

You may find more details and usage examples in Spico197/Humback .

📜 Reference

@misc{li2023selfalignment,
    title={Self-Alignment with Instruction Backtranslation},
    author={Xian Li and Ping Yu and Chunting Zhou and Timo Schick and Luke Zettlemoyer and Omer Levy and Jason Weston and Mike Lewis},
    year={2023},
    eprint={2308.06259},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}