| --- |
| license: apache-2.0 |
| datasets: |
| - timdettmers/openassistant-guanaco |
| language: |
| - en |
| base_model: |
| - meta-llama/Llama-2-7b-chat-hf |
| --- |
| |
| ## 🐋 Humpback-reproduce |
|
|
| This is a backward model _Myx_ for [Self-Alignment with Instruction Backtranslation](https://arxiv.org/pdf/2308.06259.pdf) reproduction. |
|
|
| This model (llama2 7B) is trained on the seed data ([openassistant-guanaco](https://huggingface.co/datasets/timdettmers/openassistant-guanaco) __ENGLISH DATA ONLY__) in a reversed order ((output, instruction) pairs {(yi, xi)}). |
|
|
| In other words, the model is trained by using the output to predict the instruction. |
|
|
| ## 📜 Reference |
|
|
| ```bibtex |
| @misc{li2023selfalignment, |
| title={Self-Alignment with Instruction Backtranslation}, |
| author={Xian Li and Ping Yu and Chunting Zhou and Timo Schick and Luke Zettlemoyer and Omer Levy and Jason Weston and Mike Lewis}, |
| year={2023}, |
| eprint={2308.06259}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CL} |
| } |
| ``` |
