How to compare sentences?
#9
by yalastik - opened
Is there a more advanced way to compare vectorised sentences than just with cosine/euclidean? It seems like there are transformer options which do that and it can only be done by fine-tuning and changing the target?
Did you figure this out?
Sadly, no
With this model, comparing vectors can only be done using cosine similarity/dot score/euclidean distance. If you want to use the attention mechanism from transformer models, then you might be interested in cross-encoder models: https://huggingface.co/cross-encoder, https://huggingface.co/BAAI/bge-reranker-base, https://huggingface.co/mixedbread-ai/mxbai-rerank-base-v1
- Tom Aarsen