README.md

---
license: other
license_name: sample-code-license
license_link: LICENSE
library_name: peft
pipeline_tag: image-text-to-text
---

# ViPer: Visual Personalization of Generative Models via Individual Preference Learning

*Tuning-free framework for personalized image generation*

[`Website`](https://viper.epfl.ch) | [`Paper`](https://arxiv.org/abs/2407.17365) | [`GitHub`](https://github.com/EPFL-VILAB/ViPer) | [`BibTeX`](#citation)  


We introduce **ViPer**, a method that personalizes the output of generative models to align with different users’ visual preferences for the same prompt. This is done via a one-time capture of the user’s general preferences and conditioning the generative model on them without the need for engineering detailed prompts.


## Installation
For install instructions, please see https://github.com/EPFL-VILAB/ViPer.


## Usage

This model can be loaded from Hugging Face Hub as follows:

```python
from transformers import AutoProcessor, BitsAndBytesConfig, AutoModelForVision2Seq
from peft import PeftModel

model = AutoModelForVision2Seq.from_pretrained("HuggingFaceM4/idefics2-8b")
model = PeftModel.from_pretrained(model, "EPFL-VILAB/Metric-ViPer")
```

Please see https://github.com/EPFL-VILAB/ViPer for more detailed instructions.

## Citation

If you find this repository helpful, please consider citing our work:
```
@article{ViPer,
  title={{ViPer}: Visual Personalization of Generative Models via Individual Preference Learning},
  author={Sogand Salehi and Mahdi Shafiei and Teresa Yeo and Roman Bachmann and Amir Zamir},
  journal={arXiv preprint arXiv:2407.17365},
  year={2024},
}
```

## License

Licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/sogandstorme/ViPer_Personalization/blob/main/LICENSE) for details.