Concurrent requests to same Diffusion Pipeline causes index out of bounds error ( Diffusion Pipeline 同時多筆請求會 index out of bounds )
1. Qwen-Image pipeline concurrent requests causing an index out of bounds error
Issue
When two image generation requests run concurrently using the same preloaded pipeline instance for inference (num_inference_steps=50), both requests fail around step 26 with the following error:
index 51 is out of bounds for dimension 0 with size 51
Suspected root cause
Both requests are sharing the scheduler’s internal timestep index.
When each request runs ~26 steps concurrently (26 × 2 = 52), the combined progression exceeds the scheduler’s timesteps size (50), resulting in an out-of-bounds access.
Code
import threading
from diffusers import DiffusionPipeline
import torch
self.device = "cuda"
self.torch_dtype = torch.bfloat16
self.pipe = DiffusionPipeline.from_pretrained(
"Qwen/Qwen-Image",
torch_dtype=self.torch_dtype,
local_files_only=True,
cache_dir=self.cache_dir
).to(self.device)
images = self.pipe(
prompt=prompt,
negative_prompt=negative_prompt,
width=width,
height=height,
num_inference_steps=50,
true_cfg_scale=true_cfg_scale,
num_images_per_prompt=num_images_per_prompt,
generator=generator
).images
Known workarounds
Add a lock to serialize requests
→ Safe, but eliminates concurrency
Batching requests
→ Requires request aggregation, increases latency
Multiple pipeline instances
→ VRAM usage doubles
Questions
Is the DiffusionPipeline designed to be non–thread-safe?
Are there any parameters or configurations that allow a single pipeline instance to safely handle concurrent requests?
Are there any recommended best practices for handling this type of scenario?
Log
0%| | 0/50 [00:00<?, ?it/s]
2%|▏ | 1/50 [00:00<00:18, 2.59it/s]
0%| | 0/50 [00:00<?, ?it/s]
4%|▍ | 2/50 [00:00<00:22, 2.18it/s]
2%|▏ | 1/50 [00:00<00:38, 1.28it/s]
... (omitted) ...
52%|█████▏ | 26/50 [00:20<00:19, 1.26it/s]
52%|█████▏ | 26/50 [00:20<00:19, 1.25it/s]
CRITICAL | Unexpected error: index 51 is out of bounds for dimension 0 with size 51