someone-in-the-world Claude Sonnet 4.6 commited on
Commit
a05c007
·
1 Parent(s): d66060a

Revert FA3 and torch.compile: incompatible with this GPU environment

Browse files

torch.compile falls back everywhere due to a torch.device ConstantVariable
bug in the installed PyTorch version. FA3 fails at runtime with
"no kernel image available" — the vllm-flash-attn3 kernel is not compiled
for this GPU's compute capability. Both add overhead with no benefit here.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Files changed (1) hide show
  1. app.py +1 -13
app.py CHANGED
@@ -86,19 +86,7 @@ pipe = QwenImageEditPlusPipeline.from_pretrained(
86
  _hb.set()
87
  print(f"[startup] pipeline loaded in {time.perf_counter()-_t1_load:.1f}s", flush=True)
88
 
89
- try:
90
- from qwenimage.qwen_fa3_processor import QwenDoubleStreamAttnProcessorFA3
91
- _fa3_processor = QwenDoubleStreamAttnProcessorFA3()
92
- for _block in pipe.transformer.transformer_blocks:
93
- _block.attn.processor = _fa3_processor
94
- print("[startup] FA3 attention processor enabled.", flush=True)
95
- except Exception as _fa3_err:
96
- print(f"[startup] FA3 not available ({_fa3_err}), using default attention processor.", flush=True)
97
-
98
- torch._dynamo.config.suppress_errors = True
99
- print("[startup] applying torch.compile to transformer (mode=reduce-overhead)...", flush=True)
100
- pipe.transformer = torch.compile(pipe.transformer, mode="reduce-overhead")
101
- print("[startup] torch.compile done.", flush=True)
102
 
103
  with open("examples.json") as _f:
104
  EXAMPLES_CONFIG = json.load(_f)
 
86
  _hb.set()
87
  print(f"[startup] pipeline loaded in {time.perf_counter()-_t1_load:.1f}s", flush=True)
88
 
89
+ print("[startup] using default attention processor.", flush=True)
 
 
 
 
 
 
 
 
 
 
 
 
90
 
91
  with open("examples.json") as _f:
92
  EXAMPLES_CONFIG = json.load(_f)