moose Claude Opus 4.5 commited on
Commit
07e1710
·
1 Parent(s): bbac3b5

Upgrade to Qwen-Image-Edit-2511 with Phr00t v18 transformer

Browse files

- Update base model from 2509 to 2511
- Use extracted Phr00t v18 NSFW transformer (Sneak-Moose/Qwen-Rapid-AIO-v18-NSFW-diffusers)
- Comment out next-scene LoRA for initial testing (trained on 2509)
- Update UI description and CLAUDE.md documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Files changed (2) hide show
  1. CLAUDE.md +4 -4
  2. app.py +24 -14
CLAUDE.md CHANGED
@@ -4,12 +4,12 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
4
 
5
  ## Project Overview
6
 
7
- This is a Gradio Space that implements "Next Scene" cinematic image generation using Qwen-Image-Edit-2509 with LoRA fine-tuning. The application generates visually progressive image sequences with natural cinematic transitions from frame to frame, optimized for fast 4-step inference.
8
 
9
  **Key Model Components:**
10
- - Base model: `Qwen/Qwen-Image-Edit-2509` (image editing diffusion model)
11
- - Accelerated transformer: `linoyts/Qwen-Image-Edit-Rapid-AIO` (4-step optimized variant)
12
- - LoRA adapter: `lovis93/next-scene-qwen-image-lora-2509` (cinematic progression fine-tune)
13
 
14
  ## Running the Application
15
 
 
4
 
5
  ## Project Overview
6
 
7
+ This is a Gradio Space that implements "Next Scene" cinematic image generation using Qwen-Image-Edit-2511 with LoRA fine-tuning. The application generates visually progressive image sequences with natural cinematic transitions from frame to frame, optimized for fast 4-step inference.
8
 
9
  **Key Model Components:**
10
+ - Base model: `Qwen/Qwen-Image-Edit-2511` (image editing diffusion model)
11
+ - Accelerated transformer: `Sneak-Moose/Qwen-Rapid-AIO-v18-NSFW-diffusers` (extracted from Phr00t's v18, 4-step optimized)
12
+ - LoRA adapter: `lovis93/next-scene-qwen-image-lora-2509` (cinematic progression fine-tune, trained on 2509)
13
 
14
  ## Running the Application
15
 
app.py CHANGED
@@ -93,19 +93,29 @@ def use_history_as_input(evt: gr.SelectData):
93
  dtype = torch.bfloat16
94
  device = "cuda" if torch.cuda.is_available() else "cpu"
95
 
96
- pipe = QwenImageEditPlusPipeline.from_pretrained("Qwen/Qwen-Image-Edit-2509",
97
- transformer= QwenImageTransformer2DModel.from_pretrained("linoyts/Qwen-Image-Edit-Rapid-AIO",
98
- subfolder='transformer',
99
- torch_dtype=dtype,
100
- device_map='cuda'),torch_dtype=dtype).to(device)
101
-
102
- pipe.load_lora_weights(
103
- "lovis93/next-scene-qwen-image-lora-2509",
104
- weight_name="next-scene_lora-v2-3000.safetensors", adapter_name="next-scene"
105
- )
106
- pipe.set_adapters(["next-scene"], adapter_weights=[1.])
107
- pipe.fuse_lora(adapter_names=["next-scene"], lora_scale=1.)
108
- pipe.unload_lora_weights()
 
 
 
 
 
 
 
 
 
 
109
 
110
 
111
  # Apply the same optimizations from the first version
@@ -234,7 +244,7 @@ with gr.Blocks(css=css) as demo:
234
  </div>
235
  """)
236
  gr.Markdown("""
237
- This demo uses [Qwen-Image-Edit-2509](https://huggingface.co/Qwen/Qwen-Image-Edit-2509) with [lovis93/next-scene-qwen-image-lora](https://huggingface.co/lovis93/next-scene-qwen-image-lora-2509) for cinematic image sequences with natural visual progression from frame to frame 🎥 and [linoyts/Qwen-Image-Edit-Rapid-AIO](https://huggingface.co/linoyts/Qwen-Image-Edit-Rapid-AIO) + [AoT compilation & FA3](https://huggingface.co/blog/zerogpu-aoti) for accelerated 4-step inference.
238
 
239
  Upload an image and enter your prompt to generate the next scene. The model will use your prompt exactly as provided.
240
  """)
 
93
  dtype = torch.bfloat16
94
  device = "cuda" if torch.cuda.is_available() else "cpu"
95
 
96
+ # Load Qwen-Image-Edit-2511 with Phr00t's v18 accelerated transformer (4-step inference)
97
+ pipe = QwenImageEditPlusPipeline.from_pretrained(
98
+ "Qwen/Qwen-Image-Edit-2511",
99
+ transformer=QwenImageTransformer2DModel.from_pretrained(
100
+ "Sneak-Moose/Qwen-Rapid-AIO-v18-NSFW-diffusers",
101
+ subfolder='transformer',
102
+ torch_dtype=dtype,
103
+ device_map='cuda'
104
+ ),
105
+ torch_dtype=dtype
106
+ ).to(device)
107
+
108
+ # Load next-scene LoRA for cinematic progression
109
+ # Note: This LoRA was trained on 2509, may need testing with 2511/v18
110
+ # TODO: Re-enable after testing base 2511/v18 works correctly
111
+ # pipe.load_lora_weights(
112
+ # "lovis93/next-scene-qwen-image-lora-2509",
113
+ # weight_name="next-scene_lora-v2-3000.safetensors",
114
+ # adapter_name="next-scene"
115
+ # )
116
+ # pipe.set_adapters(["next-scene"], adapter_weights=[1.])
117
+ # pipe.fuse_lora(adapter_names=["next-scene"], lora_scale=1.)
118
+ # pipe.unload_lora_weights()
119
 
120
 
121
  # Apply the same optimizations from the first version
 
244
  </div>
245
  """)
246
  gr.Markdown("""
247
+ This demo uses [Qwen-Image-Edit-2511](https://huggingface.co/Qwen/Qwen-Image-Edit-2511) with [Phr00t's Rapid-AIO v18](https://huggingface.co/Phr00t/Qwen-Image-Edit-Rapid-AIO) transformer + [lovis93/next-scene-qwen-image-lora](https://huggingface.co/lovis93/next-scene-qwen-image-lora-2509) for cinematic image sequences with natural visual progression 🎥 and [AoT compilation & FA3](https://huggingface.co/blog/zerogpu-aoti) for accelerated 4-step inference.
248
 
249
  Upload an image and enter your prompt to generate the next scene. The model will use your prompt exactly as provided.
250
  """)