Create README.md

README.md

@@ -0,0 +1,89 @@
+---
+language:
+- en
+tags:
+- bpo
+- llama
+- thudm
+inference: false
+---
+
+<h1>Black-Box Prompt Optimization: Aligning Large Language Models without Model Training</h1>
+
+- **Repository:** https://github.com/thu-coai/BPO
+- **Paper:** https://arxiv.org/abs/2311.04155
+- **Data:** https://huggingface.co/datasets/THUDM/BPO
+
+# Black-box Prompt Optimization (BPO)
+BPO is a black-box alignment technique that differs from training-based methods (like PPO or DPO). BPO only requires training of a plug-and-play model and optimizes LLMs through optimizing user inputs. Therefore, it can be used on a variety of open-source or API-based LLMs.
+
+## Model Details
+
+### Data
+Prompt优化模型由隐含人类偏好特征的prompt优化对训练得到，数据集的详细信息在这里。
+The Prompt Optimization Model is trained on prompt optimization pairs which contain human preference features. Detailed information on the dataset can be found [here](https://huggingface.co/datasets/CCCCCC/BPO).
+
+### Backbone Model
+The prompt preference optimizer is built on `Llama-2-7b-chat-hf`.
+
+### Language
+English
+
+### Performance
+
+
+| Model A| Model B | A win | tie | B win |
+|-------------|-------------|----|----|----|
+| gpt-3.5-turbo + BPO | gpt-3.5-turbo | **60.0** | 8.7 | 31.3 |
+| claude-2 + BPO | claude-2 | **57.5** | 5.0 | 37.5 |
+| llama-2-13b-chat + BPO | llama-2-70b-chat | **61.3** | 0.0 | 38.7 |
+| vicuna-13b + BPO | vicuna-13b + PPO | **52.5** | 3.7 | 43.7  |
+| vicuna-13b + BPO | vicuna-13b + DPO | **53.8** | 2.5 | 43.7 |
+| vicuna-13b + DPO + BPO | vicuna-13b + DPO | **60.0** | 2.5 | 37.5 |
+
+## Intended Use
+
+### Prompt Template 
+We adopt a prompt template as 
+```
+[INST] You are an expert prompt engineer. Please help me improve this prompt to get a more helpful and harmless response:\n{user prompt} [/INST]
+```
+
+### Inference code
+Here is an example code for inference: 
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_path = 'Your-Model-Path'
+
+prompt_template = "[INST] You are an expert prompt engineer. Please help me improve this prompt to get a more helpful and harmless response:\n{} [/INST]"
+
+model = AutoModelForCausalLM.from_pretrained(model_path).cuda()
+tokenizer = AutoTokenizer.from_pretrained(model_path)
+
+text = 'Tell me about Harry Potter'
+
+prompt = prompt_template.format(text)
+model_inputs = tokenizer(prompt, return_tensors="pt").to("cuda:0")
+output = model.generate(**model_inputs, max_new_tokens=1024, do_sample=True, top_p=0.9, temperature=0.6, num_beams=1)
+resp = tokenizer.decode(output[0], skip_special_tokens=True).split('[/INST]')[1].strip()
+
+print(resp)
+```
+See our [Github Repo](https://github.com/thu-coai/BPO/blob/main/src/infer_example.py) for more detailed usage (e.g. more aggressive optimization).
+
+
+### Other Known Limitations
+- Task coverage is not sufficient, as we only used open-source data to get about 14k optimized prompts. Clearly, it is impossible to cover a wide range of user queries, so the current model may not perform well on every prompt.
+- Due to the small ratio of long-context-based tasks and mathematical problems, the prompt optimizer underperforms when dealing with these tasks.
+
+## Citation
+If you find our model is useful in your work, please cite it with:
+```
+@article{cheng2023black,
+  title={Black-Box Prompt Optimization: Aligning Large Language Models without Model Training},
+  author={Cheng, Jiale and Liu, Xiao and Zheng, Kehan and Ke, Pei and Wang, Hongning and Dong, Yuxiao and Tang, Jie and Huang, Minlie},
+  journal={arXiv preprint arXiv:2311.04155},
+  year={2023}
+}
+```