Met a problem with qwen3.5_9b_w4a8_gptq.safetensors

#15
by hpx9527 - opened
return torch.nn.functional.scaled_dot_product_attention(q, k, v, *args, **kwargs)

RuntimeError: Expected query, key, and value to have the same dtype,
but got query.dtype: struct c10::BFloat16
key.dtype: float
and value.dtype: float instead.

Sign up or log in to comment