ardplanetary/tw_LoRA_finetune

LoRA Fine-tune and ComfyUI Workflow

This repository contains a .safetensor LoRA file and a corresponding .json ComfyUI workflow for generating portraits of a beautiful Asian woman in a specific artistic style.

---

How to Use

1. Download Files

• LoRA File: Download the .safetensor file from this repository.
• Workflow File: Download the .json file from this repository.

2. Setup

1. Place the LoRA File: Put the downloaded .safetensor file into your ComfyUI/models/loras/ directory.

2. Place the Model Files: This workflow uses the FLUX model. You will need to download the following files and place them in the specified directories within your ComfyUI/models/ folder:

   File Type	File Name	Download Location (Example)	Destination Folder
   UNET	flux1-dev.safetensors	Hugging Face Link	ComfyUI/models/flux/
   VAE	ae.sft	Hugging Face Link	ComfyUI/models/flux/
   Dual CLIP	t5xxl_fp16.safetensors	Hugging Face Link	ComfyUI/models/flux/
   Dual CLIP	clip_l.safetensors	Hugging Face Link	ComfyUI/models/flux/

3. Load the Workflow:

   • Open your ComfyUI interface.
   • Drag and drop the downloaded .json file onto the ComfyUI canvas, or use the "Load" button in the right-hand menu.
   • as API check here

3. Generate Images

Once the workflow is loaded and all the required models are in place, you can generate images.

• Trigger Word: The trigger word for this LoRA is t3ngw3i. Make sure to include this in your positive prompt.
• Prompting: The main prompt can be modified in the CLIP Text Encode (Prompt) node. The workflow is pre-configured with the prompt: t3ngw3i a portrait of a beatiful asian woman.
• Generate: Click the "Queue Prompt" button to start the image generation process.
• via Api: sample function

---

Workflow Breakdown

This ComfyUI workflow is designed to utilize the FLUX model architecture and apply a custom LoRA. Here's a breakdown of the key nodes in the provided .json file:

Model Loading

• UNETLoader (Load Diffusion Model): This node loads the main diffusion model, which in this case is flux/flux1-dev.safetensors. This is the core model that generates the image.

• DualCLIPLoader: This node loads the two text encoders required by the FLUX model: flux/t5xxl_fp16.safetensors and flux/clip_l.safetensors. These models are responsible for understanding your text prompt.

• VAELoader (Load VAE): This loads the Variational Autoencoder, flux/ae.sft, which is used to decode the generated image from latent space into a viewable format.

• LoraLoaderModelOnly: This is where the custom LoRA is applied.

  • lora_name: This should be the name of your downloaded .safetensor file (e.g., t4ngw3i.safetensors). The workflow will look for this file in your ComfyUI/models/loras/ directory.

Prompting and Guidance

• CLIPTextEncode (Prompt):

  • The first one takes your prompt. This is where you describe what you want to see in the image and include the trigger word (t3ngw3i).

• FluxGuidance: This node is specific to the FLUX model and helps guide the image generation process based on your prompt.

Output

• SaveImage: This node saves the generated image to your ComfyUI output directory.
• "filename_prefix": "discord/tw/discord_ksampler", is the current placeholder

---

Placeholders and Customization

To adapt this workflow for your own use or with different models, you will primarily need to change the file paths in the following nodes:

Node Title	Parameter to Change	Your Custom Value
Load Diffusion Model	unet_name	your_unet_model.safetensors
Load VAE	vae_name	your_vae.safetensors
DualCLIPLoader	clip_name1, clip_name2	your_clip_model_1.safetensors, your_clip_model_2.safetensors
LoraLoaderModelOnly	lora_name	your_lora.safetensors
CLIP Text Encode (Prompt)	text	"your trigger word and prompt"

By understanding this structure, you can easily modify the workflow to experiment with different models, LoRAs, and creative prompts.

To change the model to the fastest one

"12": { "inputs": { "unet_name": "flux/flux1-dev.safetensors", "weight_dtype": "fp8_e4m3fn" }, "class_type": "UNETLoader", "_meta": { "title": "Load Diffusion Model" }

  `unet_name` change to `flux1-dev-fp8-e5m2.safetensors` which should be placed on `ComfyUI\models\unet`