add IP-Adapter-FaceID-Portrait

README.md

@@ -56,9 +56,15 @@ IP-Adapter-FaceID-SDXL: An experimental SDXL version of IP-Adapter-FaceID
 ![results](./sdxl_faceid.jpg)
 </div>
 
-**Update 2024/01/17**: 
+**Update 2024/01/19**: 
+
+IP-Adapter-FaceID-Portrait: same with IP-Adapter-FaceID but for portrait generation (no lora! no controlnet!). Specifically, it accepts multiple facial images to enhance similarity (the default is 5).
+
+<div  align="center">
+
+![results](./faceid_portrait_sd15.jpg)
+</div>
 
-IP-Adapter-FaceID-PlusV2-SDXL: An experimental SDXL version of IP-Adapter-FaceID-PlusV2
 
 ## Usage
 
@@ -316,6 +322,75 @@ images = ip_model.generate(
 
 ```
 
+### IP-Adapter-FaceID-Portrait
+
+```python
+
+import cv2
+from insightface.app import FaceAnalysis
+import torch
+
+app = FaceAnalysis(name="buffalo_l", providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
+app.prepare(ctx_id=0, det_size=(640, 640))
+
+
+images = ["1.jpg", "2.jpg", "3.jpg", "4.jpg", "5.jpg"]
+
+faceid_embeds = []
+for image in images:
+    image = cv2.imread("person.jpg")
+    faces = app.get(image)
+    faceid_embeds.append(torch.from_numpy(faces[0].normed_embedding).unsqueeze(0).unsqueeze(0))
+  faceid_embeds = torch.cat(faceid_embeds, dim=1)
+```
+
+```python
+import torch
+from diffusers import StableDiffusionPipeline, DDIMScheduler, AutoencoderKL
+from PIL import Image
+
+from ip_adapter.ip_adapter_faceid_separate import IPAdapterFaceID
+
+base_model_path = "SG161222/Realistic_Vision_V4.0_noVAE"
+vae_model_path = "stabilityai/sd-vae-ft-mse"
+ip_ckpt = "ip-adapter-faceid-portrait_sd15.bin"
+device = "cuda"
+
+noise_scheduler = DDIMScheduler(
+    num_train_timesteps=1000,
+    beta_start=0.00085,
+    beta_end=0.012,
+    beta_schedule="scaled_linear",
+    clip_sample=False,
+    set_alpha_to_one=False,
+    steps_offset=1,
+)
+vae = AutoencoderKL.from_pretrained(vae_model_path).to(dtype=torch.float16)
+pipe = StableDiffusionPipeline.from_pretrained(
+    base_model_path,
+    torch_dtype=torch.float16,
+    scheduler=noise_scheduler,
+    vae=vae,
+    feature_extractor=None,
+    safety_checker=None
+)
+
+
+# load ip-adapter
+ip_model = IPAdapterFaceID(pipe, ip_ckpt, device, num_tokens=16, n_cond=5)
+
+# generate image
+prompt = "photo of a woman in red dress in a garden"
+negative_prompt = "monochrome, lowres, bad anatomy, worst quality, low quality, blurry"
+
+images = ip_model.generate(
+    prompt=prompt, negative_prompt=negative_prompt, faceid_embeds=faceid_embeds, num_samples=4, width=512, height=512, num_inference_steps=30, seed=2023
+)
+
+
+```
+
+
 
 ## Limitations and Bias
 - The models do not achieve perfect photorealism and ID consistency.