Tim419/Humpback_Myx

🐋 Humpback-reproduce

This is a backward model Myx for Self-Alignment with Instruction Backtranslation reproduction.

This model (llama2 7B) is trained on the seed data (openassistant-guanaco ENGLISH DATA ONLY) in a reversed order ((output, instruction) pairs {(yi, xi)}).

In other words, the model is trained by using the output to predict the instruction.

📜 Reference

@misc{li2023selfalignment,
    title={Self-Alignment with Instruction Backtranslation},
    author={Xian Li and Ping Yu and Chunting Zhou and Timo Schick and Luke Zettlemoyer and Omer Levy and Jason Weston and Mike Lewis},
    year={2023},
    eprint={2308.06259},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}