README.md

---
license: apache-2.0
datasets:
- timdettmers/openassistant-guanaco
language:
- en
base_model:
- meta-llama/Llama-2-7b-chat-hf
---

## 🐋 Humpback-reproduce

This is a backward model _Myx_ for [Self-Alignment with Instruction Backtranslation](https://arxiv.org/pdf/2308.06259.pdf) reproduction.

This model (llama2 7B) is trained on the seed data ([openassistant-guanaco](https://huggingface.co/datasets/timdettmers/openassistant-guanaco) __ENGLISH DATA ONLY__) in a reversed order ((output, instruction) pairs {(yi, xi)}). 

In other words, the model is trained by using the output to predict the instruction. 

## 📜 Reference

```bibtex
@misc{li2023selfalignment,
    title={Self-Alignment with Instruction Backtranslation},
    author={Xian Li and Ping Yu and Chunting Zhou and Timo Schick and Luke Zettlemoyer and Omer Levy and Jason Weston and Mike Lewis},
    year={2023},
    eprint={2308.06259},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```