Repetition Improves Language Model Embeddings
Paper • 2402.15449 • Published • 2
Feature Extraction
Transformers
Safetensors
mistral
mteb
Eval Results (legacy)
text-embeddings-inference
# Load model directly
from transformers import AutoTokenizer, AutoModel
tokenizer = AutoTokenizer.from_pretrained("jspringer/echo-mistral-7b-instruct-lasttoken")
model = AutoModel.from_pretrained("jspringer/echo-mistral-7b-instruct-lasttoken")Quick Links
Repetition Improves Language Model Embeddings
Please refer to our paper: https://arxiv.org/abs/2402.15449
And our GitHub: https://github.com/jakespringer/echo-embeddings
We provide a description of the model as well as example usage in the above links.
- Downloads last month
- 167
Spaces using jspringer/echo-mistral-7b-instruct-lasttoken 11
Paper for jspringer/echo-mistral-7b-instruct-lasttoken
Evaluation results
- accuracy on MTEB AmazonCounterfactualClassification (en)test set self-reported82.970
- ap on MTEB AmazonCounterfactualClassification (en)test set self-reported49.629
- f1 on MTEB AmazonCounterfactualClassification (en)test set self-reported77.590
- accuracy on MTEB AmazonPolarityClassificationtest set self-reported90.975
- ap on MTEB AmazonPolarityClassificationtest set self-reported87.573
- f1 on MTEB AmazonPolarityClassificationtest set self-reported90.967
- accuracy on MTEB AmazonReviewsClassification (en)test set self-reported48.708
- f1 on MTEB AmazonReviewsClassification (en)test set self-reported47.736
- map_at_1 on MTEB ArguAnatest set self-reported32.006
- map_at_10 on MTEB ArguAnatest set self-reported49.268
- map_at_100 on MTEB ArguAnatest set self-reported49.904
- map_at_1000 on MTEB ArguAnatest set self-reported49.909
- map_at_3 on MTEB ArguAnatest set self-reported44.334
- map_at_5 on MTEB ArguAnatest set self-reported47.374
- mrr_at_1 on MTEB ArguAnatest set self-reported32.788
- mrr_at_10 on MTEB ArguAnatest set self-reported49.707
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="jspringer/echo-mistral-7b-instruct-lasttoken")