FronyAI commited on
Commit
2128ea9
·
verified ·
1 Parent(s): 73dce49

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -17
README.md CHANGED
@@ -29,8 +29,9 @@ All training and data preprocessing were performed on **a single GPU (46VRAM)**,
29
 
30
  ### Model Description
31
  - **Model Type:** Sentence Transformer
32
- - **Base Model:** klue/roberta-large
33
- - **Maximum Sequence Length:** 512 tokens
 
34
  - **Output Dimensionality:** 1024 / 512 dimensions
35
  - **Similarity Function:** Cosine Similarity
36
  - **Languages:** ko, en
@@ -66,21 +67,20 @@ One group is based on a specific sports regulation PDF, for which synthetic quer
66
  The final group is a concatenation of all four aforementioned groups, providing a comprehensive mixed set.<br>
67
  The following table presents the average retrieval performance across five dataset groups.
68
 
69
- | Models | Open/Closed | Size | Accuracy@1 | Accuracy@3 | Accuracy@5 | Accuracy@10 |
70
- |--------------|-----------|-----------|-----------|------------|------------|-------------|
71
- | frony-embed-medium | **Open** | 337M | 0.6649 | **0.8040** | 0.8458 | 0.8876 |
72
- | frony-embed-medium (half dim) | Open | 337M | 0.6520 | 0.7923 | 0.8361 | 0.8796 |
73
- | frony-embed-small | Open | 111M | 0.6152 | 0.7616 | 0.8056 | 0.8559 |
74
- | frony-embed-small (half dim) | Open | 111M | 0.5988 | 0.7478 | 0.7984 | 0.8461 |
75
- | frony-embed-tiny | Open | 21M* | 0.5084 | 0.6757 | 0.7278 | 0.7845 |
76
- | frony-embed-tiny (half dim) | Open | 21M* | 0.4710 | 0.6390 | 0.6933 | 0.7596 |
77
- | bge-m3 | **Open** | 560M | 0.5852 | **0.7763** | 0.8418 | 0.8987 |
78
- | multilingual-e5-large | Open | 560M | 0.5764 | 0.7630 | 0.8267 | 0.8891 |
79
- | snowflake-arctic-embed-l-v2.0 | Open | 568M | 0.5726 | 0.7591 | 0.8232 | 0.8917 |
80
- | jina-embeddings-v3 | Open | 572M | 0.5270 | 0.7246 | 0.7953 | 0.8649 |
81
- | upstage-large | **Closed** | - | 0.6334 | **0.8527** | 0.9065 | 0.9478 |
82
- | openai-text-embedding-3-large | Closed | - | 0.4907 | 0.6617 | 0.7311 | 0.8148 |
83
- **Transformer blocks only*
84
 
85
  ## Usage
86
 
 
29
 
30
  ### Model Description
31
  - **Model Type:** Sentence Transformer
32
+ - **Base Model:** Snowflake/snowflake-arctic-embed-l-v2.0
33
+ - **Maximum Sequence Length:** 8192 tokens
34
+ > Important: The base model supports up to 8192 tokens, **but performance is only guaranteed up to 512 tokens**.
35
  - **Output Dimensionality:** 1024 / 512 dimensions
36
  - **Similarity Function:** Cosine Similarity
37
  - **Languages:** ko, en
 
67
  The final group is a concatenation of all four aforementioned groups, providing a comprehensive mixed set.<br>
68
  The following table presents the average retrieval performance across five dataset groups.
69
 
70
+ | Architecture | Open/Closed | Accuracy@1 | Accuracy@3 | Accuracy@5 | Accuracy@10 |
71
+ |--------------------------------------------------------|-----------|-----------|-----------|-----------|------------|
72
+ | FronyAI/frony-embed-medium-arctic-ko-v2.5 | Open | **0.6875** | **0.8557** | 0.9044 | 0.9410 |
73
+ | upstage-large | Closed | 0.6323 | 0.8522 | 0.9068 | 0.9459 |
74
+ | FronyAI/frony-embed-medium-arctic-ko-v2.5 (half dim) | Open | **0.6715** | **0.8446** | 0.8935 | 0.9364 |
75
+ | dragonkue/snowflake-arctic-embed-l-v2.0-ko | Open | **0.6612** | **0.8396** | 0.8931 | 0.9390 |
76
+ | FronyAI/frony-embed-medium-ko-v2 | Open | **0.6806** | **0.8376** | 0.8819 | 0.9207 |
77
+ | FronyAI/frony-embed-medium-ko-v2 (half dim) | Open | 0.6723 | 0.8275 | 0.8712 | 0.9157 |
78
+ | nlpai-lab/KURE-v1 | Open | 0.6434 | 0.8240 | 0.8788 | 0.9285 |
79
+ | BAAI/bge-m3 | Open | 0.5849 | 0.7763 | 0.8420 | 0.8985 |
80
+ | intfloat/multilingual-e5-large | Open | 0.5764 | 0.7630 | 0.8267 | 0.8891 |
81
+ | Snowflake/snowflake-arctic-embed-l-v2.0 | Open | 0.5726 | 0.7591 | 0.8232 | 0.8917 |
82
+ | jinaai/jina-embeddings-v3 | Open | 0.5270 | 0.7242 | 0.7953 | 0.8644 |
83
+ | openai-large | Closed | 0.4903 | 0.6621 | 0.7316 | 0.8149 |
 
84
 
85
  ## Usage
86