Update README.md
Browse files
README.md
CHANGED
|
@@ -92,6 +92,13 @@ The model excels at understanding natural language queries like:
|
|
| 92 |
|
| 93 |
## Performance and Limitations
|
| 94 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
### Strengths
|
| 96 |
- Excellent at understanding conversational and natural language queries
|
| 97 |
- Strong performance in e-commerce and hotel search scenarios
|
|
|
|
| 92 |
|
| 93 |
## Performance and Limitations
|
| 94 |
|
| 95 |
+
### Evaluation
|
| 96 |
+
The model's evaluation metrics are available on the [MTEB Leaderboard](https://huggingface.co/spaces/mteb/leaderboard)
|
| 97 |
+
- The model is currently by far the best embedding model under 1B parameters size and very easy to run locally on a small GPU due to it's memory size
|
| 98 |
+
- The model also is No 1. by a far margin on the [SemRel24STS](https://huggingface.co/datasets/SemRel/SemRel2024) task with an accuracy of 81.12 beating Google Gemini embedding model (second place) 73.14
|
| 99 |
+
- We noticed the model does exceptionally well with legal and news retrieval and similarity task from the MTEB leaderboard
|
| 100 |
+
|
| 101 |
+
|
| 102 |
### Strengths
|
| 103 |
- Excellent at understanding conversational and natural language queries
|
| 104 |
- Strong performance in e-commerce and hotel search scenarios
|