Hugging Face's logo

Qwen
/
Qwen3-Embedding-8B

Feature Extraction
sentence-transformers
Safetensors
Transformers
qwen3
text-generation
sentence-similarity
text-embeddings-inference
Model card Files
xet
 Community
28

Add missing `torch_dtype` in commented snippet in `README.md`

#9
by alvarobartt HF Staff - opened
base: refs/heads/main
←
from: refs/pr/9
Discussion Files changed
+1
-1
alvarobartt

Without setting the torch_dtype the snippet will fail with RuntimeError: FlashAttention only support fp16 and bf16 data type, as it defaults to torch.float32.
Add missing `torch_dtype` in commented snippet in `README.md`6ee130c9
 Update README.mdc5446774
alvarobartt

cc @littlebird13 in case you can merge this PR as well, thanks in advance 🤗
jupyter23
•
edited Jul 22, 2025

@Qwen Please merge this is needed to make it work
Ragnarok9

Still not merged and I've also encountered this issue recently...😔
vives

You can also set it to "torch_dtype":"auto"
this will reference config.json and load it as bfloat16
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment