Hugging Face's logo

SamilPwC-AXNode-GenAI
/
PwC-Embedding_expr

Sentence Similarity
sentence-transformers
Safetensors
Transformers
Korean
xlm-roberta
feature-extraction
text-embeddings-inference
Model card Files
xet
 Community
1
elplaguister commited on
Commit
6c51969
·
verified ·
1 Parent(s): 4c73846

Update README.md

Browse files

Added disclaimer and removed MTEB comparison. Please refer to the official MTEB page for all results.

Files changed (1) hide show
  1. README.md +11 -10
README.md CHANGED
@@ -13,9 +13,10 @@ tags:
13
  We trained the **PwC-Embedding-expr** model on top of the [multilingual-e5-large-instruct](https://huggingface.co/intfloat/multilingual-e5-large-instruct) embedding model.
14
  To enhance performance in Korean, we applied our curated augmentation to STS datasets and fine-tuned the E5 model using a carefully balanced ratio across datasets.
15
 
 
16
 
17
  ### To-do
18
- - [ ] MTEB Leaderboard
19
  - [ ] Technical Report
20
 
21
 
@@ -23,15 +24,15 @@ To enhance performance in Korean, we applied our curated augmentation to STS dat
23
  PwC-Embedding_expr was evaluated on the Korean subset of MTEB.
24
  A leaderboard link will be added once it is published.
25
 
26
- | Task | PwC-Embedding_expr | multilingual-e5-large | Max Result |
27
- |------------------|--------------------|-----------------------|------------|
28
- | KLUE-STS | 0.88 | 0.83 | 0.90 |
29
- | KLUE-TC | 0.73 | 0.61 | 0.73 |
30
- | Ko-StrategyQA | 0.80 | 0.80 | 0.83 |
31
- | KorSTS | 0.84 | 0.81 | 0.98 |
32
- | MIRACL-Reranking | 0.72 | 0.65 | 0.72 |
33
- | MIRACL-Retrieval | 0.65 | 0.59 | 0.72 |
34
- | **Average** | **0.77** | 0.71 | 0.81 |
35
 
36
 
37
  ## Model
 
13
  We trained the **PwC-Embedding-expr** model on top of the [multilingual-e5-large-instruct](https://huggingface.co/intfloat/multilingual-e5-large-instruct) embedding model.
14
  To enhance performance in Korean, we applied our curated augmentation to STS datasets and fine-tuned the E5 model using a carefully balanced ratio across datasets.
15
 
16
+ > ⚠️ This is an experimental model and is under continuous development.
17
 
18
  ### To-do
19
+ - [x] MTEB Leaderboard
20
  - [ ] Technical Report
21
 
22
 
 
24
  PwC-Embedding_expr was evaluated on the Korean subset of MTEB.
25
  A leaderboard link will be added once it is published.
26
 
27
+ | Task | PwC-Embedding_expr |
28
+ |------------------|--------------------|
29
+ | KLUE-STS | 0.88 |
30
+ | KLUE-TC | 0.73 |
31
+ | Ko-StrategyQA | 0.80 |
32
+ | KorSTS | 0.84 |
33
+ | MIRACL-Reranking | 0.72 |
34
+ | MIRACL-Retrieval | 0.65 |
35
+ | **Average** | **0.77** |
36
 
37
 
38
  ## Model