---
license: mit
---

# 🎨 StyQA: A Quasi-Agent Framework for Vesatile Style Transfer

<!-- [![arXiv](https://img.shields.io/badge/arXiv-2401.00000-b31b1b.svg)](https://arxiv.org/abs/2401.00000) -->
[![HuggingFace](https://img.shields.io/badge/🤗-HuggingFace-yellow.svg)](https://huggingface.co/ReyChiaro/StyQA)

![teaser](figs/teaser.jpg)

## 🔬 Introduction

StyQA is a unified quasi-agent framework for versatile style transfer tasks (pixel-level, semantic-level and continuous style transfer, etc.). This agent is attempted to be prompted with *style transfer pipeline* and normal style transfer instructions.

Typically, StyQA will conduct style-analysis $\rightarrow$ style-transfer $\rightarrow$ style-criteria pipeline, and iteratively refines the outputs based on the maximum refinement times. For convenient usage, we provide single stage calling, StyQA can run a single stage or combination of stages.

## 🚀 Quick Start

### Environment

StyQA relies on [uv](https://docs.astral.sh/uv/getting-started/) and we provided `pyproject.toml` in this repo. Note that the embedded base model relies on the newest `diffusers` which should be installed with:

```sh
uv pip install git+https://github.com/huggingface/diffusers
```

> Computational Resource: Make sure your device satisfies the computation requirements:
> - The model with `bfloat16` will comsume about 52G. If LoRA is used, each LoRA module consumes about 1G. So for better inference with at least $1024\times 1024$ pictures, $\geq 80$G is required. 

### Demos

`main.py` provides demos for continuous style transfer:

```python
@hydra.main(version_base="v1.2", config_path="configs", config_name="agent")
def main(cfgs: OmegaConf):
    prompt = "Convert the pixel colors of Picture 1 into the pixel colors of Picture 2 with strength 0.75 and then transfer the semantic style into Picture 3."
    cnt_image_path = "demos/content1.jpg"
    ref_image_paths = ["demos/pixel.jpg", "demos/semantic1.jpg"]

    agent: StyQA = instantiate(cfgs.agent)
    user_input = UserInput(
        prompt=prompt,
        cnt_image_path=cnt_image_path,
        ref_image_paths=ref_image_paths,
    )
    agent.run_pipeline(user_input)
```

Simply run

```sh
python main.py
```

and you will get the outputs like:

![output-1](figs/output1.jpg)

The agent configurations (`seed, num_inference_steps`, etc.) can be modified in `configs/agent.yaml`, or you can modify them using bash command thanks to [hydra](https://hydra.cc/).

```sh
python main.py agent.seed=1234 agent.num_inference_steps=16
```

Note that the prompt can points a specific value and style transfer task types explicitly (it is what we recommanded) or with an implicit representation such as

```sh
"Using the colors and textures from Picture 2 ... moderate strength ... as if they are in same style category ..."
```

## 🖼️ Visualization

We provide more visualization performances.

### Pixel-level style transfer

Similar to arbitrary image style transfer, Pixel-level style transfer aims to utilize the color and textures features from low level pixels.

![pixel-level](figs/pixel.jpg)

### Semantic-level style transfer

Artistic styles can be devided into different categories, semantic-level style transfer aims to re-generate content image using the same style category, which we call style-centric semantic features.

![semantic-level](figs/semantic.jpg)

### Continuous style transfer

Based on different pipeline commands, StyQA will parse them into workflow and conducts style transfer one-by-one, here we provide somes demos.

![continuous](figs/continuous.jpg)

## 📝 TODO

- [x] Open-source inference code
- [x] Deliver demos
- [ ] Uploads prepared LoRA weights
- [ ] Open-source training/fine-tuning code
- [ ] More experiments