Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/image_1_origin.jpg filter=lfs diff=lfs merge=lfs -text
+assets/image_2_origin.jpg filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,91 @@
+---
+license: apache-2.0
+---
+# LoRA Encoder (FLUX.1-Dev)
+
+This model encodes LoRA models for FLUX into embedding vectors, unlocking the capabilities of the LoRA models.
+
+Taking the LoRA model [VoidOc/F.1_Animal_Forest_LoRA](https://www.modelscope.cn/models/VoidOc/flux_animal_forest1) as an example, the LoRA encoder can be used in the following ways.
+
+## Method 1: LoRA Usage Inference
+
+Given a LoRA model, with no additional information and using an empty prompt, the LoRA encoder can directly activate the LoRA's capabilities, allowing inference of its intended use.
+
+Prompt: `""`
+
+|Without LoRA Encoder|With LoRA Encoder|
+|-|-|
+|![](./assets/image_1_origin.jpg)|![](./assets/image_1.jpg)|
+
+## Method 2: Trigger-Free Activation of LoRA Capabilities
+
+Activate the LoRA's capabilities automatically without needing to specify any trigger words.
+
+Prompt: `"a car"`
+
+|Without LoRA Encoder|With LoRA Encoder|
+|-|-|
+|![](./assets/image_2_origin.jpg)|![](./assets/image_2.jpg)|
+
+## Method 3: LoRA Strength Control
+
+An additional parameter `scale` is provided to control the influence of the LoRA on the generated image.
+
+In the example below, the prompt is `"a cat"`. When `scale=1`, the LoRA exerts maximum influence, resulting in an image showing both a character from Animal Crossing and a cat. When `scale=0.5`, the LoRA's influence is reduced, producing an image of a cat character from Animal Crossing. The optimal `scale` value depends on the specific LoRA model; we recommend using larger values for character-based LoRAs and smaller values for style-based LoRAs.
+
+Prompt: `"a cat"`
+
+|`scale=1`|`scale=0.5`|
+|-|-|
+|![](./assets/image_3.jpg)|![](./assets/image_3_scale.jpg)|
+
+## Inference Code
+
+```
+git clone https://github.com/modelscope/DiffSynth-Studio.git  
+cd DiffSynth-Studio
+pip install -e .
+```
+
+```python
+import torch
+from diffsynth.pipelines.flux_image_new import FluxImagePipeline, ModelConfig
+
+
+pipe = FluxImagePipeline.from_pretrained(
+    torch_dtype=torch.bfloat16,
+    device="cuda",
+    model_configs=[
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="flux1-dev.safetensors"),
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="text_encoder/model.safetensors"),
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="text_encoder_2/"),
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="ae.safetensors"),
+        ModelConfig(model_id="DiffSynth-Studio/LoRA-Encoder-FLUX.1-Dev", origin_file_pattern="model.safetensors"),
+    ],
+)
+pipe.enable_lora_magic()
+
+lora = ModelConfig(model_id="VoidOc/flux_animal_forest1", origin_file_pattern="20.safetensors")
+pipe.load_lora(pipe.dit, lora, hotload=True) # Use `pipe.clear_lora()` to drop the loaded LoRA.
+
+# Empty prompt can automatically activate LoRA capabilities.
+image = pipe(prompt="", seed=0, lora_encoder_inputs=lora)
+image.save("image_1.jpg")
+
+image = pipe(prompt="", seed=0)
+image.save("image_1_origin.jpg")
+
+# Prompt without trigger words can also activate LoRA capabilities.
+image = pipe(prompt="a car", seed=0, lora_encoder_inputs=lora)
+image.save("image_2.jpg")
+
+image = pipe(prompt="a car", seed=0)
+image.save("image_2_origin.jpg")
+
+# Adjust the activation intensity through the scale parameter.
+image = pipe(prompt="a cat", seed=0, lora_encoder_inputs=lora, lora_encoder_scale=1.0)
+image.save("image_3.jpg")
+
+image = pipe(prompt="a cat", seed=0, lora_encoder_inputs=lora, lora_encoder_scale=0.5)
+image.save("image_3_scale.jpg")
+```

README_from_modelscope.md

@@ -0,0 +1,119 @@
+---
+frameworks:
+- Pytorch
+license: Apache License 2.0
+tasks:
+- text-to-image-synthesis
+
+#model-type:
+##如 gpt、phi、llama、chatglm、baichuan 等
+#- gpt
+
+#domain:
+##如 nlp、cv、audio、multi-modal
+#- nlp
+
+#language:
+##语言代码列表 https://help.aliyun.com/document_detail/215387.html?spm=a2c4g.11186623.0.0.9f8d7467kni6Aa
+#- cn 
+
+#metrics:
+##如 CIDEr、Blue、ROUGE 等
+#- CIDEr
+
+#tags:
+##各种自定义，包括 pretrained、fine-tuned、instruction-tuned、RL-tuned 等训练方法和其他
+#- pretrained
+
+#tools:
+##如 vllm、fastchat、llamacpp、AdaSeq 等
+#- vllm
+---
+# LoRA 编码器（FLUX.1-Dev）
+
+本模型可以将 FLUX 模型的 LoRA 模型编码为 Embedding 向量，激发出 LoRA 模型的能力。
+
+以 LoRA 模型 [VoidOc/F.1_动物森友会LoRA](https://www.modelscope.cn/models/VoidOc/flux_animal_forest1) 为例，LoRA 编码器有以下几种使用方法。
+
+## 使用方法1：LoRA 用途推断
+
+给定一个 LoRA 模型，在没有任何额外信息的条件下，使用空提示词可以直接激发 LoRA 模型的能力，进而推断出 LoRA 的用途。
+
+提示词：`""`
+
+|不使用 LoRA 编码器|使用 LoRA 编码器|
+|-|-|
+|![](./assets/image_1_origin.jpg)|![](./assets/image_1.jpg)|
+
+## 使用方法2：免触发词激发 LoRA 能力
+
+无需填写触发词，即可自动激发 LoRA 的能力。
+
+提示词：`"a car"`
+
+|不使用 LoRA 编码器|使用 LoRA 编码器|
+|-|-|
+|![](./assets/image_2_origin.jpg)|![](./assets/image_2.jpg)|
+
+## 使用方法3：LoRA 强度控制
+
+我们预留了一个额外的参数 `scale`，控制 LoRA 对模型生成图像的影响大小。
+
+在下面的例子中，提示词为“a cat”，当 `scale=1` 时，LoRA 强度为最大，画面中生成了动物森友会中的角色和一只猫；当 `scale=0.5` 时，LoRA 强度被减弱，画面中生成了动物森友会中的猫猫角色。`scale` 的最优数值与 LoRA 模型本身有关，我们建议在角色 LoRA 上使用较大的数值，在风格 LoRA 上使用较小的数值。
+
+提示词：`"a cat"`
+
+|`scale=1`|`scale=0.5`|
+|-|-|
+|![](./assets/image_3.jpg)|![](./assets/image_3_scale.jpg)|
+
+## 推理代码
+
+```
+git clone https://github.com/modelscope/DiffSynth-Studio.git  
+cd DiffSynth-Studio
+pip install -e .
+```
+
+```python
+import torch
+from diffsynth.pipelines.flux_image_new import FluxImagePipeline, ModelConfig
+
+
+pipe = FluxImagePipeline.from_pretrained(
+    torch_dtype=torch.bfloat16,
+    device="cuda",
+    model_configs=[
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="flux1-dev.safetensors"),
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="text_encoder/model.safetensors"),
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="text_encoder_2/"),
+        ModelConfig(model_id="black-forest-labs/FLUX.1-dev", origin_file_pattern="ae.safetensors"),
+        ModelConfig(model_id="DiffSynth-Studio/LoRA-Encoder-FLUX.1-Dev", origin_file_pattern="model.safetensors"),
+    ],
+)
+pipe.enable_lora_magic()
+
+lora = ModelConfig(model_id="VoidOc/flux_animal_forest1", origin_file_pattern="20.safetensors")
+pipe.load_lora(pipe.dit, lora, hotload=True) # Use `pipe.clear_lora()` to drop the loaded LoRA.
+
+# Empty prompt can automatically activate LoRA capabilities.
+image = pipe(prompt="", seed=0, lora_encoder_inputs=lora)
+image.save("image_1.jpg")
+
+image = pipe(prompt="", seed=0)
+image.save("image_1_origin.jpg")
+
+# Prompt without trigger words can also activate LoRA capabilities.
+image = pipe(prompt="a car", seed=0, lora_encoder_inputs=lora)
+image.save("image_2.jpg")
+
+image = pipe(prompt="a car", seed=0,)
+image.save("image_2_origin.jpg")
+
+# Adjust the activation intensity through the scale parameter.
+image = pipe(prompt="a cat", seed=0, lora_encoder_inputs=lora, lora_encoder_scale=1.0)
+image.save("image_3.jpg")
+
+image = pipe(prompt="a cat", seed=0, lora_encoder_inputs=lora, lora_encoder_scale=0.5)
+image.save("image_3_scale.jpg")
+```

configuration.json

@@ -0,0 +1 @@
+{"framework":"Pytorch","task":"text-to-image-synthesis"}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a123efe19d4edcc50c38508148bd0e6af154d8358daeb33669e764730984b9f6
+size 1402316728