simlang commited on
Commit
99a3f39
·
1 Parent(s): c908756

fix: redo timings with kv caching enabled + add end-to-end plots

Browse files
README.md CHANGED
@@ -70,6 +70,17 @@ model, so the pipeline, text encoder, and VAE stay unchanged.
70
  > - The first distillation removing need for prompt upsampling into the LoRA, thus significantly reducing end-to-end generation.
71
  > The v0.1 adapters remain available.
72
 
 
 
 
 
 
 
 
 
 
 
 
73
  ## 6-step variants
74
 
75
  Pick **6-steps** for faithful base-model behavior, or **6-steps-upsampled** for
@@ -220,11 +231,6 @@ pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
220
 
221
 
222
  <table>
223
- <tr>
224
- <td>
225
- <img src="assets/P-Q-Image-2.1-t2i-1024-all.png" alt="Text-to-image generation latency at 1024 pixels across base, 8-step, 6-step, 6-step-upsampled, and 5-step adapters" width="100%">
226
- </td>
227
- </tr>
228
  <tr>
229
  <td>
230
  <img src="assets/P-Q-Image-2.1-v02-generation.png" alt="Six-step generation latency at 1024 and 2048 pixels" width="100%">
@@ -237,10 +243,17 @@ pipe.scheduler = FlowMatchEulerDiscreteScheduler.from_config(
237
  </tr>
238
  </table>
239
 
240
- One NVIDIA H100 80GB, BF16, batch size 1. Median of 3 after one warmup, seed 42.
 
 
241
  Generation includes prompt encoding, reference-image processing, denoising, and
242
  VAE decode. It excludes loading, warmup, fixture prep, saving, and network.
243
- No CFG, compilation, CPU offload, or diffusion KV caching.
 
 
 
 
 
244
 
245
  Prompt upsampling is timed separately with
246
  [`Qwen-Image-2.1-PE-T2I`](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I)
@@ -249,7 +262,7 @@ It covers request prep through parsed text, not image generation. vLLM runs
249
  eager, without prefix or image-processor caching, and with a fresh image ID so
250
  vision encoding is always counted. Autoregressive KV cache stays on.
251
 
252
- Text-to-image uses the official Qwen model-card prompt. Editing uses a fixed
253
  prompt and 1–3 reference images, square-cropped and resized. These cases are
254
  not matched to the v0.1 charts, and adding the two medians is not an
255
  end-to-end time.
 
70
  > - The first distillation removing need for prompt upsampling into the LoRA, thus significantly reducing end-to-end generation.
71
  > The v0.1 adapters remain available.
72
 
73
+ <table>
74
+ <tr>
75
+ <td width="50%">
76
+ <img src="assets/P-Q-Image-2.1-t2i-1024-end-to-end.png" alt="Estimated end-to-end text-to-image latency at 1024 × 1024" width="100%">
77
+ </td>
78
+ <td width="50%">
79
+ <img src="assets/P-Q-Image-2.1-i2i-1024-end-to-end.png" alt="Estimated end-to-end editing latency at 1024 × 1024 with one input image" width="100%">
80
+ </td>
81
+ </tr>
82
+ </table>
83
+
84
  ## 6-step variants
85
 
86
  Pick **6-steps** for faithful base-model behavior, or **6-steps-upsampled** for
 
231
 
232
 
233
  <table>
 
 
 
 
 
234
  <tr>
235
  <td>
236
  <img src="assets/P-Q-Image-2.1-v02-generation.png" alt="Six-step generation latency at 1024 and 2048 pixels" width="100%">
 
243
  </tr>
244
  </table>
245
 
246
+ One NVIDIA H100 80GB, BF16, batch size 1. v0.2 generation timings
247
+ report the median of six measured requests pooled across both adapters (three
248
+ per adapter), excluding one warmup per adapter and case.
249
  Generation includes prompt encoding, reference-image processing, denoising, and
250
  VAE decode. It excludes loading, warmup, fixture prep, saving, and network.
251
+ No CFG, compilation, or CPU offload. **Diffusion KV caching is enabled**, except
252
+ for the case marked with an asterisk below.
253
+
254
+ \* **2048 × 2048 editing with 3 input images:** KV caching was disabled due to
255
+ memory constraints on the H100 80GB. The 2048 multi-image cases use expandable
256
+ CUDA memory segments (`PYTORCH_ALLOC_CONF=expandable_segments:True`).
257
 
258
  Prompt upsampling is timed separately with
259
  [`Qwen-Image-2.1-PE-T2I`](https://huggingface.co/Qwen/Qwen-Image-2.1-PE-T2I)
 
262
  eager, without prefix or image-processor caching, and with a fresh image ID so
263
  vision encoding is always counted. Autoregressive KV cache stays on.
264
 
265
+ Text-to-image uses a fixed neon-sign prompt. Editing uses a fixed
266
  prompt and 1–3 reference images, square-cropped and resized. These cases are
267
  not matched to the v0.1 charts, and adding the two medians is not an
268
  end-to-end time.
assets/P-Q-Image-2.1-i2i-1024-end-to-end.png ADDED

Git LFS Details

  • SHA256: 575cb47b782367b284ce04545fb107e57a6b154c7501c64b5bca1d508380171f
  • Pointer size: 131 Bytes
  • Size of remote file: 221 kB
assets/P-Q-Image-2.1-t2i-1024-all.png CHANGED

Git LFS Details

  • SHA256: e991203220650ebaf7764cb1b0987616dea511d80dd5bb530341626997fa9dc7
  • Pointer size: 131 Bytes
  • Size of remote file: 137 kB

Git LFS Details

  • SHA256: 2600fffac13e33fca1c4629a31cedb09a3ced9ddf49df2dd70d3a9415c998a52
  • Pointer size: 131 Bytes
  • Size of remote file: 166 kB
assets/P-Q-Image-2.1-t2i-1024-end-to-end.png ADDED

Git LFS Details

  • SHA256: f66bea020903efdf12e67fa9ad9ba1fcb59157059f2f80e2865b9d178e627645
  • Pointer size: 131 Bytes
  • Size of remote file: 223 kB
assets/P-Q-Image-2.1-v02-generation.png CHANGED

Git LFS Details

  • SHA256: e39d8b02ef6315d782a93c90edb821c6a1a577acc6f564f86c2fabc8624ec31c
  • Pointer size: 131 Bytes
  • Size of remote file: 126 kB

Git LFS Details

  • SHA256: 000eaec12906ba71a07942522b9e6db5e8624d35e9f42dc67d53a9813d76ebbd
  • Pointer size: 131 Bytes
  • Size of remote file: 138 kB