AisingioroHao0 commited on
Commit
fcada9f
·
1 Parent(s): 86d44dc

update README

Browse files
README.assets/artistic_portrait_generation_examples.jpg ADDED

Git LFS Details

  • SHA256: 90c22813689f9ed1e49072f456d2536965d426757f580fda36ebb9839a25dcce
  • Pointer size: 132 Bytes
  • Size of remote file: 1.32 MB
README.assets/artistic_portrait_generation_pipeline.jpg ADDED

Git LFS Details

  • SHA256: a989558a5439bc4f0fb382dc0e471ff915ea3da008127ec760f143246af22302
  • Pointer size: 131 Bytes
  • Size of remote file: 994 kB
README.assets/comparison_with_existing_methods.jpg ADDED

Git LFS Details

  • SHA256: f8e418f9364b09ddba121e522b3ab7c53c824fa80a996b4967536708d851af80
  • Pointer size: 132 Bytes
  • Size of remote file: 1.78 MB
README.assets/stylize_controlnet_parameter_visualization.jpg ADDED

Git LFS Details

  • SHA256: da7917e2aabf9a242e4bd2bea25a1d7a48ff9116dc684cd34866aef4df9e0382
  • Pointer size: 132 Bytes
  • Size of remote file: 1.11 MB
README.md CHANGED
@@ -14,21 +14,51 @@ IP Adapter Art is a specialized version that uses a professional style encoder.
14
 
15
  [![**IP Adapter Art Demo**](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1kV7q3Gzr8GPG9cChdDQ5ncCx84TYjuu3?usp=sharing) can be used to conduct experiments directly.
16
 
17
- For local experiments, please refer to a [demo](https://github.com/aihao2000/IP-Adapter-Artist/blob/main/ip_adapter_art_sdxl_demo.ipynb).
18
 
19
  Local experiments require a basic torch environment and dependencies:
20
 
21
  ```
22
- pip install diffusers
23
- pip install transformers
 
24
  pip install git+https://github.com/openai/CLIP.git
25
- pip install git+https://github.com/aihao2000/IP-Adapter-Art.git
26
  ```
27
 
28
- ## More Examples
 
 
 
 
 
 
 
 
 
 
 
 
 
 
29
 
30
  ![image-20240808001612810](./README.assets/more_examples.png)
31
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
32
 
33
  ## Citation
34
 
@@ -41,4 +71,7 @@ pip install git+https://github.com/aihao2000/IP-Adapter-Art.git
41
  journal = {GitHub repository},
42
  howpublished = {\url{https://github.com/aihao2000/IP-Adapter-Art}}
43
  }
44
- ```
 
 
 
 
14
 
15
  [![**IP Adapter Art Demo**](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1kV7q3Gzr8GPG9cChdDQ5ncCx84TYjuu3?usp=sharing) can be used to conduct experiments directly.
16
 
17
+ For local experiments, please refer to a [demo](https://github.com/aihao2000/IP-Adapter-Art/blob/main/artistic_portrait_gen.ipynb).
18
 
19
  Local experiments require a basic torch environment and dependencies:
20
 
21
  ```
22
+ conda create -n artadapter python=3.10
23
+ conda activate artadapter
24
+ pip install -r requirements.txt
25
  pip install git+https://github.com/openai/CLIP.git
26
+ pip install -e .
27
  ```
28
 
29
+ ## Comparison with Existing Style Control Methods in Diffusion Models
30
+
31
+ Evaluation using [StyleBench](https://github.com/open-mmlab/StyleShot) style images. Image quality is evaluated using [improved aesthetic predictor](https://github.com/christophschuhmann/improved-aesthetic-predictor)
32
+
33
+ | | CLIP Style Similarity | CSD Style Similarity | CLIP Text Alignment | Image Quality | Average |
34
+ | --------------------- | --------------------- | -------------------- | ------------------- | ------------- | --------- |
35
+ | DEADiff | 61.99 | 43.54 | 20.82 | 60.76 | 46.78 |
36
+ | StyleShot | 63.01 | 52.40 | 18.93 | 55.54 | 47.47 |
37
+ | Instant Style | 65.39 | 58.39 | 21.09 | 60.62 | 51.37 |
38
+ | **Art-Adapter(ours)** | **67.03** | **65.02** | 20.25 | **62.23** | **53.63** |
39
+
40
+ ![image](./README.assets/comparison_with_existing_methods.jpg)
41
+
42
+
43
+ ## Examples of Text-guided Stylized Generation
44
 
45
  ![image-20240808001612810](./README.assets/more_examples.png)
46
 
47
+ ## Artistic Portrait Generation
48
+
49
+ ### Pipeline
50
+
51
+ We built an artistic portrait generation pipeline using Art-Adapter, PuLID, and ControlNet. The structure is shown in the figure below.
52
+
53
+ ![image-20240808001612811](./README.assets/artistic_portrait_generation_pipeline.jpg)
54
+
55
+ ### Examples
56
+
57
+ ![image-20240808001612812](./README.assets/artistic_portrait_generation_examples.jpg)
58
+
59
+ ## Stylize ControlNet Parameter Visualization
60
+
61
+ ![image-20240808001612813](./README.assets/stylize_controlnet_parameter_visualization.jpg)
62
 
63
  ## Citation
64
 
 
71
  journal = {GitHub repository},
72
  howpublished = {\url{https://github.com/aihao2000/IP-Adapter-Art}}
73
  }
74
+ ```
75
+
76
+ ## Acknowledgements
77
+