Hugging Face's logo

BAAI
/
bge-m3

Sentence Similarity
sentence-transformers
PyTorch
ONNX
xlm-roberta
feature-extraction
Eval Results
text-embeddings-inference
Model card Files
xet
 Community
144

Issue: Semantic Similarity Score

#55
by linhphanff - opened
Discussion
linhphanff
•
edited May 21, 2024

Firstly, I want to thank you for the great job you guys have done with the extremely powerful model, I have an issue right now. When I calculate the similarity between two identical sentences using dense vectors, why is the score 0.9995 (should be 1)? Can you explain?
Sentence 1: "We recommend"
Sentence 2: "We recommend"
Shitao
 Beijing Academy of Artificial Intelligence org
•
edited May 21, 2024

Thanks for your attention to our work!
If you use fp16, numerical overflow is likely to occur, which may cause slight differences. However, the error is very small and does not significantly affect the ranking results.
If you load model with transformers package, you need to set model.eval(), otherwise the dropout operation will cause two different embeddings for the same sentence.
linhphanff

Thanks for your answer !!

Sign up or log in to comment