Add Key Highlights, Model List and Experimental Results sections to all Octen model READMEs
Browse files
README.md
CHANGED
|
@@ -24,6 +24,76 @@ Octen-Embedding-8B-INT8 is a text embedding model designed for semantic search a
|
|
| 24 |
|
| 25 |
**Quantization**: This is an INT8 quantized version using bitsandbytes. INT8 quantization significantly reduces memory footprint (~50% smaller), making it suitable for deployment on resource-constrained environments. Note that while memory usage is reduced, inference speed may not necessarily improve and could be slightly slower than the BF16 version on some hardware.
|
| 26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
## Model Details
|
| 28 |
|
| 29 |
- **Base Model**: [Qwen/Qwen3-Embedding-8B](https://huggingface.co/Qwen/Qwen3-Embedding-8B)
|
|
|
|
| 24 |
|
| 25 |
**Quantization**: This is an INT8 quantized version using bitsandbytes. INT8 quantization significantly reduces memory footprint (~50% smaller), making it suitable for deployment on resource-constrained environments. Note that while memory usage is reduced, inference speed may not necessarily improve and could be slightly slower than the BF16 version on some hardware.
|
| 26 |
|
| 27 |
+
## Key Highlights
|
| 28 |
+
|
| 29 |
+
### 🥇 RTEB Leaderboard Champion (as of January 12, 2026)
|
| 30 |
+
- **Octen-Embedding-8B ranks #1 on the [RTEB Leaderboard](https://huggingface.co/spaces/mteb/leaderboard)** with Mean (Task) score of **0.8045**
|
| 31 |
+
- Excellent performance on both Public (0.7953) and Private (0.8157) datasets
|
| 32 |
+
- Demonstrates true generalization capability without overfitting to public benchmarks
|
| 33 |
+
|
| 34 |
+
### Industry-Oriented Vertical Domain Expertise
|
| 35 |
+
- **Legal**: Legal document retrieval
|
| 36 |
+
- **Finance**: Financial reports, Q&A, and personal finance content
|
| 37 |
+
- **Healthcare**: Medical Q&A, clinical dialogues, and health consultations
|
| 38 |
+
- **Code**: Programming problems, code search, and SQL queries
|
| 39 |
+
|
| 40 |
+
### Ultra-Long Context Support
|
| 41 |
+
- Supports up to **32,768 tokens** context length
|
| 42 |
+
- Suitable for processing long documents in legal, healthcare, and other domains
|
| 43 |
+
- High-dimensional embedding space for rich semantic representation
|
| 44 |
+
|
| 45 |
+
### Multilingual Capability
|
| 46 |
+
- Supports **100+ languages**
|
| 47 |
+
- Includes various programming languages
|
| 48 |
+
- Strong multilingual, cross-lingual, and code retrieval capabilities
|
| 49 |
+
|
| 50 |
+
---
|
| 51 |
+
|
| 52 |
+
## Open Source Model List
|
| 53 |
+
|
| 54 |
+
| Model Type | Model | Size | Max Tokens | Embedding Dimensions | HuggingFace Link |
|
| 55 |
+
|------------|-------|------|------------|---------------------|------------------|
|
| 56 |
+
| Text Embedding | [Octen-Embedding-0.6B](https://huggingface.co/Octen/Octen-Embedding-0.6B) | 0.6B | 32,768 | 1024 | ✅ Available |
|
| 57 |
+
| Text Embedding | [Octen-Embedding-4B](https://huggingface.co/Octen/Octen-Embedding-4B) | 4.0B | 32,768 | 2560 | ✅ Available |
|
| 58 |
+
| Text Embedding | [Octen-Embedding-8B](https://huggingface.co/Octen/Octen-Embedding-8B) | 7.6B | 32,768 | 4096 | ✅ Available |
|
| 59 |
+
|
| 60 |
+
**Model Family Design**:
|
| 61 |
+
- **Octen-Embedding-8B**: Best performance, RTEB #1, for high-precision retrieval
|
| 62 |
+
- **Octen-Embedding-4B**: Best in 4B category, balanced performance and efficiency
|
| 63 |
+
- **Octen-Embedding-0.6B**: Lightweight deployment, suitable for edge devices and resource-constrained environments
|
| 64 |
+
|
| 65 |
+
---
|
| 66 |
+
|
| 67 |
+
## Experimental Results
|
| 68 |
+
|
| 69 |
+
### RTEB Leaderboard (Overall Performance)
|
| 70 |
+
|
| 71 |
+
| Model | Embedding Dim | Max Tokens | Mean (Public) | Mean (Private) | Mean (Task) |
|
| 72 |
+
|-------|---------------|------------|---------------|----------------|-------------|
|
| 73 |
+
| **Octen-Embedding-8B** | **4096** | **32768** | **0.7953** | **0.8157** | **0.8045** |
|
| 74 |
+
| voyage-3-large | 1024 | 32000 | 0.7434 | 0.8277 | 0.7812 |
|
| 75 |
+
| gemini-embedding-001 | 3072 | 2048 | 0.7218 | 0.8075 | 0.7602 |
|
| 76 |
+
| **Octen-Embedding-4B** | **2560** | **32768** | **0.7747** | **0.7942** | **0.7834** |
|
| 77 |
+
| MoD-Embedding | 2560 | 32768 | 0.7642 | 0.7900 | 0.7758 |
|
| 78 |
+
| Qwen3-Embedding-8B | 4096 | 32768 | 0.7310 | 0.7838 | 0.7547 |
|
| 79 |
+
| **Octen-Embedding-0.6B** | **1024** | **32768** | **0.7241** | **-** | **-** |
|
| 80 |
+
| voyage-3.5 | 1024 | 32000 | 0.7139 | 0.8102 | 0.7571 |
|
| 81 |
+
| Cohere-embed-v4.0 | 1536 | 128000 | 0.6534 | 0.7943 | 0.7166 |
|
| 82 |
+
| jina-embeddings-v4 | 2048 | 32768 | 0.6652 | 0.7664 | 0.7105 |
|
| 83 |
+
| GritLM-7B | 4096 | 32768 | 0.6187 | 0.7385 | 0.6724 |
|
| 84 |
+
| text-embedding-3-large | 3072 | 8191 | 0.6110 | 0.7130 | 0.6567 |
|
| 85 |
+
| e5-mistral-7b-instruct | 4096 | 32768 | 0.5090 | 0.7091 | 0.5987 |
|
| 86 |
+
| NV-Embed-v2 | 4096 | 32768 | 0.5805 | 0.6691 | 0.6203 |
|
| 87 |
+
| snowflake-arctic-embed-l-v2.0 | 1024 | 8192 | 0.5395 | 0.7079 | 0.6150 |
|
| 88 |
+
| multilingual-e5-large-instruct | 1024 | 514 | 0.5478 | 0.6859 | 0.6097 |
|
| 89 |
+
| gte-multilingual-base | 768 | 8192 | 0.5291 | 0.6697 | 0.5921 |
|
| 90 |
+
| text-embedding-3-small | 1536 | 8191 | 0.5260 | 0.6630 | 0.5874 |
|
| 91 |
+
| bge-m3 | 1024 | 8194 | 0.5216 | 0.6726 | 0.5893 |
|
| 92 |
+
| Qwen3-Embedding-4B | 2560 | 32768 | - | 0.7711 | - |
|
| 93 |
+
| Qwen3-Embedding-0.6B | 1024 | 32768 | - | 0.7117 | - |
|
| 94 |
+
|
| 95 |
+
---
|
| 96 |
+
|
| 97 |
## Model Details
|
| 98 |
|
| 99 |
- **Base Model**: [Qwen/Qwen3-Embedding-8B](https://huggingface.co/Qwen/Qwen3-Embedding-8B)
|