|
|
--- |
|
|
tags: |
|
|
- text-to-image |
|
|
- lora |
|
|
- diffusers |
|
|
- template:diffusion-lora |
|
|
- flux |
|
|
widget: |
|
|
- text: >- |
|
|
[FOUR-VIEWS] a red desk lamp from multiple views;[TOP-LEFT] This photo shows |
|
|
a 45-degree angle of desk lamp;[TOP-RIGHT] This photo shows a high-angle |
|
|
shot of the lamp; [BOTTOM-LEFT] Here is a side view shot of lamp; |
|
|
[BOTTOM-RIGHT] The back view of the desk lamp. |
|
|
output: |
|
|
url: images/example_qevsnjb3v.png |
|
|
- text: >- |
|
|
[FOUR-VIEWS] This set of four images show different angles of an IKEA white |
|
|
bed ; [TOP-LEFT] This photo shows a side view of the bed; [TOP-RIGHT] This |
|
|
photo shows the left view of the bed; [BOTTOM-LEFT] This photo shows a front |
|
|
view of the bed; [BOTTOM-RIGHT] This photo shows a back view of the bed." |
|
|
output: |
|
|
url: images/example_n5u06nx5j.png |
|
|
- text: >- |
|
|
[FOUR-VIEWS] This set of four images show different angles of a golden |
|
|
motorbike; [TOP-LEFT] This photo shows a full frontal view of the motorbike; |
|
|
[TOP-RIGHT] This photo shows a 45 degree angle of the motorbike; |
|
|
[BOTTOM-LEFT] This photo shows a front view of the motorbike; [BOTTOM-RIGHT] |
|
|
This photo shows the motorbike from above. |
|
|
output: |
|
|
url: images/example_jg3yw7dcl.png |
|
|
- text: >- |
|
|
[FOUR-VIEWS] a bedroom from multiple views;[TOP-LEFT] This photo shows a |
|
|
45-degree angle of the bedroom;[TOP-RIGHT] This photo shows a high-angle |
|
|
shot of the bedroom; [BOTTOM-LEFT] Here is a side view shot of bedroom; |
|
|
[BOTTOM-RIGHT] A low angle view of the bedroom. |
|
|
output: |
|
|
url: images/example_w9qva3imf.png |
|
|
- text: >- |
|
|
[FOUR-VIEWS] this photo set shows a cute pug dog from multiple |
|
|
angles;[TOP-LEFT] This photo shows a 45-degree angle of the pug ;[TOP-RIGHT] |
|
|
This photo shows a high-angle shot of the pug; [BOTTOM-LEFT] Here is a side |
|
|
view shot of the pug.[BOTTOM-RIGHT] A low angle view of the pug.. |
|
|
output: |
|
|
url: images/example_cujunw6xh.png |
|
|
base_model: black-forest-labs/FLUX.1-dev |
|
|
instance_prompt: '[FOUR-VIEWS]' |
|
|
license: apache-2.0 |
|
|
pipeline_tag: text-to-image |
|
|
language: |
|
|
- en |
|
|
|
|
|
--- |
|
|
# fourviews-incontext-lora |
|
|
|
|
|
<Gallery /> |
|
|
|
|
|
## Model description |
|
|
|
|
|
base_model: black-forest-labs;FLUX-1-dev |
|
|
- 2x2-grid |
|
|
- in-context |
|
|
model_type: lora |
|
|
|
|
|
Inspired by [In-Context-LoRA](https://github.com/ali-vilab/In-Context-LoRA), this project aims to generate four multi-view images of the same scene or object simultaneously. By using flux with the multiview-incontext-lora, we can divide the images into portions to obtain novel views. |
|
|
|
|
|
> **_NOTE:_** This is a beta release of the model. The consistency between views may not be perfect, and the model might sometimes generate views that don't perfectly align or maintain exact object positions across viewpoints. |
|
|
|
|
|
|
|
|
# [FOUR-VIEWS-IMAGES] 2 × 2-Grid LoRA |
|
|
|
|
|
**Base:** FLUX-1-dev |
|
|
**Images:** 126 custom image-text composites resized or padded to 512x512 from [MVImgNET](https://github.com/GAP-LAB-CUHK-SZ/MVImgNet/tree/main). |
|
|
The first image of the blue bag is from the dataset  |
|
|
|
|
|
**Steps:** 1000 |
|
|
**LoRA Rank:** 8 |
|
|
**Trigger token:**[FOUR-VIEWS]; |
|
|
|
|
|
```python |
|
|
import torch |
|
|
from diffusers import FluxPipeline |
|
|
|
|
|
pipeline = FluxPipeline.from_pretrained( |
|
|
"black-forest-labs/FLUX.1-dev", |
|
|
torch_dtype=torch.bfloat16, |
|
|
) |
|
|
pipeline.load_lora_weights( |
|
|
"rmsandu/fourviews-incontext-lora", |
|
|
weight_name="4views.safetensors", |
|
|
) |
|
|
pipeline.fuse_lora() |
|
|
|
|
|
prompt = f"[FOUR-VIEWS] This set of four images shows a jade dragon statue different viewpoints. [TOP-LEFT] This photo shows a 45-degree angle of jade statue;[TOP-RIGHT] This photo shows a high-angle shot of the statue; [BOTTOM-LEFT] Here is a side view shot of the statue; [BOTTOM-RIGHT] The back view of the statue." |
|
|
image_height = 512 |
|
|
image_width = 512 |
|
|
output = pipeline( |
|
|
prompt=prompt, |
|
|
height=int(image_height), |
|
|
width=int(image_width), |
|
|
num_inference_steps=30, |
|
|
guidance_scale=3.5, |
|
|
).images[0] |
|
|
|
|
|
output.save("fourview-incontext-beta.png") |
|
|
|
|
|
``` |
|
|
|
|
|
## Trigger words |
|
|
|
|
|
You should use `[FOUR-VIEWS]` to trigger the image generation. |
|
|
|
|
|
|
|
|
# Download model |
|
|
|
|
|
Weights for this model are available in Safetensors format. |
|
|
|
|
|
[Download](/rmsandu/fourviews-incontext-lora/tree/main) them in the Files & versions tab. |
