Instructions to use Ashmotv/transformation_lora with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Ashmotv/transformation_lora with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("minimax-h3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("Ashmotv/transformation_lora") prompt = "Tr@nsf0rmation_style" input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png") image = pipe(image=input_image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
Tr@nsf0rmation_style β MiniMax H3 Video Transformation LoRA
This LoRA is fine-tuned on the MiniMax H3 (Hailuo) diffusion transformer architecture to generate dynamic, seamless visual morphing and transformation transitions between two images or scenes. It guides the model to fluidly deconstruct and reassemble individual elements, textures, outlines, and subjects from the initial frame into the final composition.
π¬ Video Showcase
Sample 01: Scene Transformation
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
Sample 02: Full Subject Transformation
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
Sample 03: Multi-Element Scene Reassembly
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another. The bald person in grey T-shirt and blue jeans from <Picture 1> Tr@nsf0rmation into the Girl in <Picture 2>. The person in the Yellow shirt from <Picture 1> Tr@nsf0rmation into the yellow grass from <Picture 2>. The person in the green shirt from <Picture 1> Tr@nsf0rmation into the blank background in <Picture 2>.
Sample 04: Visual Transformation
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
Sample 05: Fluid Transition
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
Sample 06: Semantic Morphing
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
Sample 07: Complex Subject Swap
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
Sample 08: Pencil Stroke Evolution
Prompt: [Shot 1] black pencil outlines from Picture 1 Tr@nsf0rmation into Picture 2 with individual pencil strokes undergoing Tr@nsf0rmation into one another.
Sample 09: Dynamic Line Dissolve
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
Sample 10: High-Fidelity Flow
Prompt: [Shot 1] Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another.
π Usage Guide
Trigger Word & Prompt Structure
- Main Trigger Word:
Tr@nsf0rmation_style - Action Connector:
Tr@nsf0rmation into
The LoRA operates best when describing the starting state (<Picture 1>) transitioning into the final state (<Picture 2>), explicitly calling out how subjects, materials, or individual visual elements transform.
Recommended Prompt Template:
integrated_multimodal_description: [Shot 1] Tr@nsf0rmation_style, Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another. [Subject A from <Picture 1>] Tr@nsf0rmation into [Subject B from <Picture 2>].
Example Prompts:
- Multi-element correspondence:
integrated_multimodal_description: [Shot 1] Tr@nsf0rmation_style, Picture 1 Tr@nsf0rmation into Picture 2 with individual elements and subjects Tr@nsf0rmation into one another. The bald person in grey T-shirt and blue jeans from <Picture 1> Tr@nsf0rmation into the Girl in <Picture 2>. The person in the Yellow shirt from <Picture 1> Tr@nsf0rmation into the yellow grass from <Picture 2>. - Pencil & line-art transformation:
integrated_multimodal_description: [Shot 1] Tr@nsf0rmation_style, black pencil outlines from Picture 1 Tr@nsf0rmation into Picture 2 with individual pencil strokes undergoing Tr@nsf0rmation into one another. - Direct object transformation:
Tr@nsf0rmation_style, a ripe red tomato with green leaves Tr@nsf0rmation into a cartoon boy with brown hair in a cap and vest holding a rake over his shoulder.
βοΈ Recommended Inference Settings
| Parameter | Recommended Value | Notes |
|---|---|---|
| Base Model | MiniMax H3 FL2VA / Ref2V | e.g. minimax_h3_fl2va_pruned_int8_convrot.safetensors |
| LoRA Strength (st2500) | 1.0 β 1.2 |
Step 2500 checkpoint: fully converged transformation effect |
| LoRA Strength (st1600) | 1.0 β 1.4 |
Step 1600 checkpoint: softer, slightly lighter stylization |
| Stacked LoRA Setup | st2500 @ 1.2 + st1600 @ 1.4 |
Used in the showcase generations for maximum morphing adherence |
| Sampler | res_multistep or euler |
ResMultistep provides stable temporal transitions |
| Scheduler | simple |
Flow-match scheduler |
| Steps | 12 (with DMD turbo) / 25β30 (standard) |
Works well with 8-step/12-step distilled turbo adapters |
| Sigma Shift | shift_video = 8.0, shift_audio = 3.0 |
MiniMax H3 custom sigma shift node |
| Resolution | 852x480, 1024x576, or upscaled to 1080p |
Compatible with 3D Latent Upscaler (minimax_h3_latent_upscaler_3d_bf16) |
π¦ Checkpoints Included
Transformation_style_c1-st2500_comfyui.safetensors: Trained to 2500 steps. Strongest transition cohesion and element-level morphing.Transformation_style_c1-st1600_comfyui.safetensors: Trained to 1600 steps. Intermediate checkpoint ideal for subtle blend transitions or when combining with other style LoRAs.
π οΈ ComfyUI Integration
- Place the LoRA files inside your ComfyUI models directory:
ComfyUI/models/loras/ - Connect a
LoraLoaderModelOnlynode after your MiniMax H3 UNet loader (minimax_h3_fl2va_pruned_int8_convrot.safetensors). - Set
strength_modelbetween1.0and1.2. - Provide
<Picture 1>(start frame) and<Picture 2>(target frame) into your First-and-Last-frame conditioning nodes. - In your prompt node, format your description using
Tr@nsf0rmation_styleandTr@nsf0rmation into.
- Downloads last month
- 72