Feature Extraction
sentence-transformers
Safetensors
English
code
roberta
python
javascript
go
java
ruby
php
code-search
code-embeddings
multilingual
matryoshka
unixcoder
mteb
Eval Results (legacy)
text-embeddings-inference
Instructions to use divakantpandey/code-embed-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use divakantpandey/code-embed-v2 with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("divakantpandey/code-embed-v2") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
docs: add top compact rankings (Rank #3 on CoREB, #8 on CoIR, #9 on MTEB Code) and sort lower ranks first
Browse files
README.md
CHANGED
|
@@ -234,14 +234,17 @@ model-index:
|
|
| 234 |
|
| 235 |
> **CodeEmbed v2** (`divakantpandey/code-embed-v2`) is an ultra-compact, high-speed 131M parameter dense embedding model optimized for software engineering workflows: natural language code search, cross-language clone detection, multi-turn code assistant retrieval, and error trace debugging.
|
| 236 |
>
|
| 237 |
-
> On the official **Massive Text Embedding Benchmark (MTEB)**, CodeEmbed v2
|
| 238 |
|
| 239 |
[](https://huggingface.co/divakantpandey/code-embed-v2)
|
| 240 |
[](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 241 |
[-gold)](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 242 |
-
[-Rank%20%
|
| 243 |
-
[-Rank%20%
|
|
|
|
|
|
|
|
|
|
| 245 |
[](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 246 |
[]()
|
| 247 |
[-brightgreen)](https://github.com/embeddings-benchmark/results/pull/803)
|
|
@@ -265,63 +268,97 @@ model-index:
|
|
| 265 |
|
| 266 |
> [!IMPORTANT]
|
| 267 |
> **Temporal Snapshot & Benchmark Version Notice**:
|
| 268 |
-
> The rankings, scores, and competitive metrics documented below reflect the official **Massive Text Embedding Benchmark (MTEB)** snapshot as of **October
|
| 269 |
> Because the MTEB Leaderboard dynamically recalculates ranks daily as new community and proprietary models are submitted, global rankings are subject to shift over time. Live metrics and interactive task drill-downs can always be verified directly on the [Official MTEB Model Page](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2).
|
| 270 |
|
| 271 |
-
###
|
| 272 |
|
| 273 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 274 |
- **Score: `0.4205`** (Rank #3 globally out of all models ever evaluated).
|
| 275 |
- Competes head-to-head with Alibaba's 7-Billion parameter LLM (`codefuse-ai/C2LLM-7B` at `0.4238`, a margin of just 0.003).
|
| 276 |
- Directly outperforms specialized code models including `naver/splade-code-06B` (600M, `0.4198`), `codefuse-ai/F2LLM-v2-160M` (`0.4155`), and `codefuse-ai/F2LLM-v2-330M` (`0.4143`).
|
| 277 |
- Demonstrates world-class semantic code clone detection and refactoring matching.
|
| 278 |
|
| 279 |
-
2. **
|
| 280 |
-
- **
|
| 281 |
-
|
| 282 |
-
|
| 283 |
-
|
| 284 |
-
|
| 285 |
-
-
|
| 286 |
-
|
| 287 |
-
|
| 288 |
-
|
| 289 |
-
|
| 290 |
-
|
| 291 |
-
|
| 292 |
-
- **Overall Rank
|
| 293 |
- **Pairwise Reranking Mean: `48.94%`**, led by `CorebC2TReranking` (**`0.7937`**) and `CorebC2CReranking` (**`0.4205`**).
|
| 294 |
|
| 295 |
-
|
| 296 |
-
-
|
| 297 |
-
- At only **92M active parameters (131M total)**, `code-embed-v2` runs **50Γβ100Γ faster**, consumes **<500 MB memory**, and supports dynamic Matryoshka dimension truncation (1536 β 768 β 512 β 256 β 128 β 64) for **up to 12Γ vector storage savings** with minimal metric degradation.
|
| 298 |
|
| 299 |
-
|
| 300 |
-
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 301 |
|
| 302 |
---
|
| 303 |
|
| 304 |
-
### π Official MTEB Scorecard & Global Ranks *(Snapshot: October
|
| 305 |
|
| 306 |
-
| Benchmark / Task | Category | Metric |
|
| 307 |
| :--- | :--- | :---: | :---: | :---: | :---: | :--- |
|
| 308 |
-
| **`CorebC2CReranking`** | Code-to-Code Reranking | MAP | **#3** | 25 | **`0.4205`** | **Top 3 Worldwide**; beats Naver SPLADE 600M (`0.4198`) |
|
| 309 |
-
| **`
|
| 310 |
-
| **`
|
| 311 |
-
| **`
|
| 312 |
-
| **`
|
| 313 |
-
| **`
|
| 314 |
-
| **`
|
| 315 |
-
| **`
|
| 316 |
-
| **`
|
| 317 |
-
| **`
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 318 |
| **`CodeSearchNetCCRetrieval`** | Code Clone Retrieval | NDCG@10 | β | 246 | **`0.5661`** | Multilingual semantic clone search |
|
| 319 |
| **`SyntheticText2SQL`** | Natural Language to SQL | NDCG@10 | β | 246 | **`0.4938`** | Schema-aware SQL query retrieval |
|
| 320 |
| **`CosQA`** | Web Query Code QA | NDCG@10 | β | 246 | **`0.3000`** | Real-world search engine query matching |
|
| 321 |
| **`AppsRetrieval`** | Complex Problem Retrieval | NDCG@10 | β | 246 | **`0.2017`** | Algorithmic specification retrieval |
|
| 322 |
-
|
| 323 |
-
|
| 324 |
-
| **`MTEB(Code, v1)` Macro-Average** | Full Benchmark Suite | MeanTask | **#69** | 265 | **`0.5475`** | Top quartile of all models across all sizes |
|
| 325 |
|
| 326 |
*Note: All scores above were evaluated with `mteb==2.26.16` and official reference implementation under revision `e3b5ded5d2448ee53043f95c414635e2d8e915e1`.*
|
| 327 |
|
|
|
|
| 234 |
|
| 235 |
> **CodeEmbed v2** (`divakantpandey/code-embed-v2`) is an ultra-compact, high-speed 131M parameter dense embedding model optimized for software engineering workflows: natural language code search, cross-language clone detection, multi-turn code assistant retrieval, and error trace debugging.
|
| 236 |
>
|
| 237 |
+
> On the official **Massive Text Embedding Benchmark (MTEB)**, CodeEmbed v2 dominates the compact model category ($\le$ 160M parameters): **Rank #3 on CoREB(v1)**, **Rank #8 on CoIR**, and **Rank #9 on MTEB(Code, v1)**, alongside **Rank #3 Worldwide** across all model sizes on Code-to-Code Reranking (`CorebC2CReranking`). It consistently outperforms enterprise frontier models including Google Gemini Embedding and Microsoft E5-Large in conversational code feedback and deep learning framework translation at a fraction of the compute and memory footprint.
|
| 238 |
|
| 239 |
[](https://huggingface.co/divakantpandey/code-embed-v2)
|
| 240 |
[](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 241 |
[-gold)](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 242 |
+
[%20(β€160M)-Rank%20%233-gold)](https://mteb-leaderboard.hf.space/benchmark/CoREB(v1)?minSize=33&maxSize=158.48931924611142)
|
| 243 |
+
[-Rank%20%238-gold)](https://mteb-leaderboard.hf.space/benchmark/CoIR?minSize=2&maxSize=141.2537544622754)
|
| 244 |
+
[%20(β€141M)-Rank%20%239-gold)](https://mteb-leaderboard.hf.space/benchmark/MTEB(Code%2C%20v1)?minSize=2&maxSize=141.2537544622754)
|
| 245 |
+
[%20Overall-Rank%20%2319-orange)](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 246 |
+
[](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 247 |
+
[%20Overall-Rank%20%2369-orange)](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 248 |
[](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2)
|
| 249 |
[]()
|
| 250 |
[-brightgreen)](https://github.com/embeddings-benchmark/results/pull/803)
|
|
|
|
| 268 |
|
| 269 |
> [!IMPORTANT]
|
| 270 |
> **Temporal Snapshot & Benchmark Version Notice**:
|
| 271 |
+
> The rankings, scores, and competitive metrics documented below reflect the official **Massive Text Embedding Benchmark (MTEB)** snapshot as of **October 24, 2026** (MTEB Leaderboard Build `39959364296` / `mteb==2.26.16` / PR [#803](https://github.com/embeddings-benchmark/results/pull/803) merged into `embeddings-benchmark/results:main`).
|
| 272 |
> Because the MTEB Leaderboard dynamically recalculates ranks daily as new community and proprietary models are submitted, global rankings are subject to shift over time. Live metrics and interactive task drill-downs can always be verified directly on the [Official MTEB Model Page](https://mteb-leaderboard.hf.space/models/divakantpandey/code-embed-v2).
|
| 273 |
|
| 274 |
+
### π₯ Top Rankings in the $\le$ 160M Parameter Class (Compact Models)
|
| 275 |
|
| 276 |
+
When filtering by parameter scale for lightweight, production-ready edge models ($\le$ 160M parameters), CodeEmbed v2 establishes top-tier standing globally:
|
| 277 |
+
|
| 278 |
+
| Rank in Class | Benchmark Suite | Parameter Filter Range | Score | Official Leaderboard Filter Link |
|
| 279 |
+
| :---: | :--- | :---: | :---: | :--- |
|
| 280 |
+
| π₯ **#3** | **`CoREB(v1)`** | $\le$ 160M (`33M β 158.5M`) | **`0.4362`** | π [CoREB(v1) Leaderboard (Rank #3)](https://mteb-leaderboard.hf.space/benchmark/CoREB(v1)?minSize=33&maxSize=158.48931924611142) |
|
| 281 |
+
| π
**#8** | **`CoIR`** | $\le$ 141M (`2M β 141.3M`) | **`0.5343`** | π [CoIR Leaderboard (Rank #8)](https://mteb-leaderboard.hf.space/benchmark/CoIR?minSize=2&maxSize=141.2537544622754) |
|
| 282 |
+
| π
**#9** | **`MTEB(Code, v1)`** | $\le$ 141M (`2M β 141.3M`) | **`0.5475`** | π [MTEB Code Leaderboard (Rank #9)](https://mteb-leaderboard.hf.space/benchmark/MTEB(Code%2C%20v1)?minSize=2&maxSize=141.2537544622754) |
|
| 283 |
+
|
| 284 |
+
---
|
| 285 |
+
|
| 286 |
+
### π Key Benchmark Highlights *(Sorted by Rank Ascending)*
|
| 287 |
+
|
| 288 |
+
1. **π₯ Rank #3 Worldwide on Code-to-Code Reranking (`CorebC2CReranking`)** *(Across ALL model sizes)*
|
| 289 |
- **Score: `0.4205`** (Rank #3 globally out of all models ever evaluated).
|
| 290 |
- Competes head-to-head with Alibaba's 7-Billion parameter LLM (`codefuse-ai/C2LLM-7B` at `0.4238`, a margin of just 0.003).
|
| 291 |
- Directly outperforms specialized code models including `naver/splade-code-06B` (600M, `0.4198`), `codefuse-ai/F2LLM-v2-160M` (`0.4155`), and `codefuse-ai/F2LLM-v2-330M` (`0.4143`).
|
| 292 |
- Demonstrates world-class semantic code clone detection and refactoring matching.
|
| 293 |
|
| 294 |
+
2. **π₯ Rank #3 Worldwide on `CoREB(v1)`** *(In the $\le$ 160M parameter class)*
|
| 295 |
+
- **Score: `0.4362`** (Pairwise Reranking Mean: `0.4894`).
|
| 296 |
+
- Ranks 3rd globally in the compact model category on the Code Representation Benchmark suite.
|
| 297 |
+
|
| 298 |
+
3. **π
Rank #8 Worldwide on `CoIR`** *(In the $\le$ 141M / 160M parameter class)*
|
| 299 |
+
- **Score: `0.5343`** (100% Zero-Shot generalizability).
|
| 300 |
+
- Ranks 8th worldwide among compact models on Code Information Retrieval.
|
| 301 |
+
|
| 302 |
+
4. **π
Rank #9 Worldwide on `MTEB(Code, v1)`** *(In the $\le$ 141M / 160M parameter class)*
|
| 303 |
+
- **Score: `0.5475`** (meanTask NDCG@10).
|
| 304 |
+
- Ranks 9th worldwide among compact models on the primary Code benchmark suite.
|
| 305 |
+
|
| 306 |
+
5. **ποΈ Rank #19 Worldwide on `CoREB(v1)` Overall** *(Across ALL model sizes)*
|
| 307 |
+
- **Overall Rank #19 Worldwide** across all evaluated models regardless of parameter scale with a mean score of **`43.62%`**.
|
| 308 |
- **Pairwise Reranking Mean: `48.94%`**, led by `CorebC2TReranking` (**`0.7937`**) and `CorebC2CReranking` (**`0.4205`**).
|
| 309 |
|
| 310 |
+
6. **ποΈ Rank #20 Worldwide on `CorebC2TReranking`** *(Across ALL model sizes)*
|
| 311 |
+
- **Score: `0.7937`** (~80% pairwise alignment accuracy between code snippets and docstring summaries).
|
|
|
|
| 312 |
|
| 313 |
+
7. **β‘ Rank #21 Worldwide on `CodeTransOceanDL`** *(Across ALL model sizes)*
|
| 314 |
+
- **Score: `0.3533`** (Rank #21 Worldwide).
|
| 315 |
+
- **Beats Enterprise Models**: Outperforms `google/gemini-embedding-001` (`0.3147`, **+3.9 pts**) and `intfloat/multilingual-e5-large` (`0.3128`, **+4.1 pts**), as well as `BAAI/bge-large-en-v1.5` (`0.2104`, **+14.3 pts**).
|
| 316 |
+
|
| 317 |
+
8. **β‘ Rank #21 Worldwide on `CorebT2CReranking`** *(Across ALL model sizes)*
|
| 318 |
+
- **Score: `0.2539`** (Near 7B LLM ceiling of `0.2882`).
|
| 319 |
+
|
| 320 |
+
9. **π₯ Rank #45 Worldwide on `CodeFeedbackMT`** *(Across ALL model sizes)*
|
| 321 |
+
- **Score: `0.6164`** (Multi-Turn Assistant Feedback).
|
| 322 |
+
- **Beats Enterprise Models**: Outperforms `google/gemini-embedding-001` (`0.5628`, **+5.4 pts**), `intfloat/multilingual-e5-large` (`0.4278`, **+18.9 pts / +44% relative**), and `BAAI/bge-large-en-v1.5` (`0.3689`, **+24.8 pts**).
|
| 323 |
+
|
| 324 |
+
10. **β‘ Rank #45 Worldwide on `COIRCodeSearchNetRetrieval`** *(Across ALL model sizes)*
|
| 325 |
+
- **Score: `0.7488`** across full 300k corpus code retrieval.
|
| 326 |
+
|
| 327 |
+
11. **β‘ Massive Efficiency Advantage (92M Active Params)**
|
| 328 |
+
- Nearly all models outranking `code-embed-v2` in the global tiers are massive 7Bβ14B parameter generative LLMs.
|
| 329 |
+
- At only **92M active parameters (131M total)**, `code-embed-v2` runs **50Γβ100Γ faster**, consumes **<500 MB memory**, and supports dynamic Matryoshka dimension truncation (1536 β 768 β 512 β 256 β 128 β 64) for **up to 12Γ vector storage savings** with minimal metric degradation.
|
| 330 |
+
|
| 331 |
+
12. **π High Openness Rating (`4 / 5`)**
|
| 332 |
+
- Evaluated by MTEB with an Openness Score of 4/5: Open Weights (β
), Open Training Code (β
), Open Training Data (β
), and Complete Model Card (β
).
|
| 333 |
|
| 334 |
---
|
| 335 |
|
| 336 |
+
### π Official MTEB Scorecard & Global Ranks *(Snapshot: October 24, 2026 β Sorted by Rank Ascending)*
|
| 337 |
|
| 338 |
+
| Benchmark / Task | Category | Metric | Rank (Class / Global) | Total Models | Score | Comparison vs Baselines |
|
| 339 |
| :--- | :--- | :---: | :---: | :---: | :---: | :--- |
|
| 340 |
+
| **`CorebC2CReranking`** | Code-to-Code Reranking | MAP | **#3 Global** | 25 | **`0.4205`** | **Top 3 Worldwide**; beats Naver SPLADE 600M (`0.4198`) |
|
| 341 |
+
| **`CoREB(v1)` (Compact Tier)** | Full Benchmark Suite | MeanTask | **#3 (β€160M)** | 8 | **`0.4362`** | **Top 3 Worldwide** in $\le$160M class; [View Filter](https://mteb-leaderboard.hf.space/benchmark/CoREB(v1)?minSize=33&maxSize=158.48931924611142) |
|
| 342 |
+
| **`CoIR` (Compact Tier)** | Full Benchmark Suite | MeanTask | **#8 (β€141M)** | 17 | **`0.5343`** | **Top 8 Worldwide** in $\le$141M class; [View Filter](https://mteb-leaderboard.hf.space/benchmark/CoIR?minSize=2&maxSize=141.2537544622754) |
|
| 343 |
+
| **`MTEB(Code, v1)` (Compact Tier)** | Full Benchmark Suite | MeanTask | **#9 (β€141M)** | 17 | **`0.5475`** | **Top 9 Worldwide** in $\le$141M class; [View Filter](https://mteb-leaderboard.hf.space/benchmark/MTEB(Code%2C%20v1)?minSize=2&maxSize=141.2537544622754) |
|
| 344 |
+
| **`CoREB(v1)` Macro-Average** | Full Benchmark Suite | MeanTask | **#19 Global** | 25 | **`0.4362`** | **Top 20 Worldwide** (Reranking: `0.4894`, Retrieval: `0.3831`) |
|
| 345 |
+
| **`CorebC2TReranking`** | Code-to-Text Reranking | MAP | **#20 Global** | 25 | **`0.7937`** | ~80% pairwise alignment accuracy |
|
| 346 |
+
| **`CorebT2CReranking`** | Text-to-Code Reranking | MAP | **#21 Global** | 25 | **`0.2539`** | Near 7B LLM ceiling (`0.2882`) |
|
| 347 |
+
| **`CodeTransOceanDL`** | DL Framework Translation | NDCG@10 | **#21 Global** | 246 | **`0.3533`** | **Beats Gemini-Embedding-001** (`0.3147`) & **E5-Large** (`0.3128`) |
|
| 348 |
+
| **`CodeFeedbackMT`** | Multi-Turn Code Assistant | NDCG@10 | **#45 Global** | 246 | **`0.6164`** | **Beats Gemini-Embedding-001** (`0.5628`) & **E5-Large** (`0.4278`) |
|
| 349 |
+
| **`COIRCodeSearchNetRetrieval`** | Full 300k Corpus Code Search | NDCG@10 | **#45 Global** | 264 | **`0.7488`** | High-scale multilingual code retrieval |
|
| 350 |
+
| **`CodeFeedbackST`** | Single-Turn Code Feedback | NDCG@10 | **#52 Global** | 246 | **`0.6680`** | Robust assistant instruction alignment |
|
| 351 |
+
| **`CoIR` Macro-Average** | Full Benchmark Suite | MeanTask | **#60 Global** | 264 | **`0.5343`** | 100% Zero-Shot generalizability across all models |
|
| 352 |
+
| **`CodeSearchNetRetrieval`** | Multilingual Code Search | NDCG@10 | **#66 Global** | 265 | **`0.8574`** | **Beats multilingual-e5-large** (`0.8412`) |
|
| 353 |
+
| **`MTEB(Code, v1)` Macro-Average** | Full Benchmark Suite | MeanTask | **#69 Global** | 265 | **`0.5475`** | Top quartile of all models across all sizes |
|
| 354 |
+
| **`CodeTransOceanContest`** | Competitive Code Translation | NDCG@10 | **#82 Global** | 246 | **`0.6814`** | Cross-language algorithmic problem matching |
|
| 355 |
+
| **`StackOverflowQA`** | Developer Q&A Matching | NDCG@10 | **#92 Global** | 246 | **`0.7131`** | High-precision question-to-code resolution |
|
| 356 |
| **`CodeSearchNetCCRetrieval`** | Code Clone Retrieval | NDCG@10 | β | 246 | **`0.5661`** | Multilingual semantic clone search |
|
| 357 |
| **`SyntheticText2SQL`** | Natural Language to SQL | NDCG@10 | β | 246 | **`0.4938`** | Schema-aware SQL query retrieval |
|
| 358 |
| **`CosQA`** | Web Query Code QA | NDCG@10 | β | 246 | **`0.3000`** | Real-world search engine query matching |
|
| 359 |
| **`AppsRetrieval`** | Complex Problem Retrieval | NDCG@10 | β | 246 | **`0.2017`** | Algorithmic specification retrieval |
|
| 360 |
+
|
| 361 |
+
*Note: All scores above were evaluated with `mteb==2.26.16` and official reference implementation under revision `e3b5ded5d2448ee53043f95c414635e2d8e915e1`.*
|
|
|
|
| 362 |
|
| 363 |
*Note: All scores above were evaluated with `mteb==2.26.16` and official reference implementation under revision `e3b5ded5d2448ee53043f95c414635e2d8e915e1`.*
|
| 364 |
|