Update README.md

README.md

@@ -26,51 +26,25 @@ datasets:
 base_model: pkshatech/GLuCoSE-base-ja
 license: apache-2.0
 ---
+# GLuCoSE v2
 
-# SentenceTransformer
+This model is a general Japanese text embedding model, excelling in retrieval tasks. It can run on CPU and is designed to measure semantic similarity between sentences, as well as to function as a retrieval system for searching passages based on queries.
 
-This is a [sentence-transformers](https://www.SBERT.net) model trained. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+During inference, the prefix "query: " or "passage: " is required. Please check the Usage section for details.
 
-## Model Details
-The model is based on [GLuCoSE](https://huggingface.co/pkshatech/GLuCoSE-base-ja) and additionally fine-tuned.
-Fine-tuning consists of the following steps.
+## Model Description
 
-**Step 1: Ensemble distillation**
-
-- The embedded representation was distilled using  [E5-mistral](https://huggingface.co/intfloat/e5-mistral-7b-instruct), [gte-Qwen2](https://huggingface.co/Alibaba-NLP/gte-Qwen2-7B-instruct), and [mE5-large](https://huggingface.co/intfloat/multilingual-e5-large) as teacher models.
-
-**Step 2: Contrastive learning**
-  
-- Triples were created from [JSNLI](https://nlp.ist.i.kyoto-u.ac.jp/?%E6%97%A5%E6%9C%AC%E8%AA%9ESNLI%28JSNLI%29%E3%83%87%E3%83%BC%E3%82%BF%E3%82%BB%E3%83%83%E3%83%88), [MNLI](https://huggingface.co/datasets/MoritzLaurer/multilingual-NLI-26lang-2mil7), [PAWS-X](https://huggingface.co/datasets/paws-x), [JSeM](https://github.com/DaisukeBekki/JSeM) and [Mr.TyDi](https://huggingface.co/datasets/castorini/mr-tydi) and used for training.
-- This training aimed to improve the overall performance as a sentence embedding model.
+The model is based on [GLuCoSE](https://huggingface.co/pkshatech/GLuCoSE-base-ja) and fine-tuned through distillation using several large-scale embedding models and multi-stage contrastive learning.
 
-**Step 3: Search-specific contrastive learning**
-  
-- In order to make the model more robust to the retrieval task, additional two-stage training with QA and question-answer data was conducted.
-- In the first stage, the synthetic dataset [auto-wiki-qa](https://huggingface.co/datasets/cl-nagoya/auto-wiki-qa)  was used for training,
-while in the second stage, [Japanese Wikipedia Human Retrieval](https://huggingface.co/datasets/hpprc/emb)
-, [Mr.TyDi](https://huggingface.co/datasets/hpprc/emb), 
-[MIRACL](https://huggingface.co/datasets/hpprc/emb), 
-[JQaRA](https://huggingface.co/datasets/hotchpotch/JQaRA), 
-[MQA](https://huggingface.co/datasets/hpprc/mqa-ja), 
-[Quiz Works](https://huggingface.co/datasets/hpprc/emb) and 
-[Quiz No Mori](https://huggingface.co/datasets/hpprc/emb) were used.
-### Model Description
-- **Model Type:** Sentence Transformer
-<!-- - **Base model:** [Unknown](https://huggingface.co/unknown) -->
 - **Maximum Sequence Length:** 512 tokens
 - **Output Dimensionality:** 768 tokens
 - **Similarity Function:** Cosine Similarity
-<!-- - **Training Dataset:** Unknown -->
-<!-- - **Language:** Unknown -->
-<!-- - **License:** Unknown -->
-
 
 ## Usage
 
 ### Direct Usage (Sentence Transformers)
 
-You can perform inference using SentenceTransformers with the following code:
+You can perform inference using SentenceTransformer with the following code:
 
 ```python
 from sentence_transformers import SentenceTransformer
@@ -98,6 +72,7 @@ print(similarities)
 # [0.6050, 1.0000, 0.5018, 0.6815],
 # [0.4341, 0.5018, 1.0000, 0.7534],
 # [0.5537, 0.6815, 0.7534, 1.0000]]
+
 ```
 
 ### Direct Usage (Transformers)
@@ -142,10 +117,30 @@ print(similarities)
 # [0.6050, 1.0000, 0.5018, 0.6815],
 # [0.4341, 0.5018, 1.0000, 0.7534],
 # [0.5537, 0.6815, 0.7534, 1.0000]]
+
 ```
 
+## Training Details
+
+The fine-tuning of GLuCoSE v2 is carried out through the following steps:
+
+**Step 1: Ensemble distillation**
+
+- The embedded representation was distilled using [E5-mistral](https://huggingface.co/intfloat/e5-mistral-7b-instruct), [gte-Qwen2](https://huggingface.co/Alibaba-NLP/gte-Qwen2-7B-instruct), and [mE5-large](https://huggingface.co/intfloat/multilingual-e5-large) as teacher models.
+
+**Step 2: Contrastive learning**
+
+- Triplets were created from [JSNLI](https://nlp.ist.i.kyoto-u.ac.jp/?%E6%97%A5%E6%9C%AC%E8%AA%9ESNLI%28JSNLI%29%E3%83%87%E3%83%BC%E3%82%BF%E3%82%BB%E3%83%83%E3%83%88), [MNLI](https://huggingface.co/datasets/MoritzLaurer/multilingual-NLI-26lang-2mil7), [PAWS-X](https://huggingface.co/datasets/paws-x), [JSeM](https://github.com/DaisukeBekki/JSeM) and [Mr.TyDi](https://huggingface.co/datasets/castorini/mr-tydi) and used for training.
+- This training aimed to improve the overall performance as a sentence embedding model.
+
+**Step 3: Search-specific contrastive learning**
+
+- In order to make the model more robust to the retrieval task, additional two-stage training with QA and retrieval task was conducted.
+- In the first stage, the synthetic dataset [auto-wiki-qa](https://huggingface.co/datasets/cl-nagoya/auto-wiki-qa) was used for training,
+while in the second stage, [JQaRA](https://huggingface.co/datasets/hotchpotch/JQaRA), [MQA](https://huggingface.co/datasets/hpprc/mqa-ja), [Japanese Wikipedia Human Retrieval, Mr.TyDi,MIRACL, Quiz Works and Quiz No Mor](https://huggingface.co/datasets/hpprc/emb)i were used.
 
 <!--
+
 ### Downstream Usage (Sentence Transformers)
 
 You can finetune this model on your own dataset.
@@ -156,19 +151,21 @@ You can finetune this model on your own dataset.
 -->
 
 <!--
+
 ### Out-of-Scope Use
 
 *List how the model may foreseeably be misused and address what users ought not to do with the model.*
 -->
 
-
 <!--
+
 ## Bias, Risks and Limitations
 
 *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
 -->
 
 <!--
+
 ### Recommendations
 
 *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
@@ -176,68 +173,48 @@ You can finetune this model on your own dataset.
 
 ## Benchmarks
 
-### Retieval
+### Retrieval
+
 Evaluated with [MIRACL-ja](https://huggingface.co/datasets/miracl/miracl), [JQARA](https://huggingface.co/datasets/hotchpotch/JQaRA) , [JaCWIR](https://huggingface.co/datasets/hotchpotch/JaCWIR) and [MLDR-ja](https://huggingface.co/datasets/Shitao/MLDR).
 
 | Model | Size | MIRACL<br>Recall@5 | JQaRA<br>nDCG@10 | JaCWIR<br>MAP@10 | MLDR<br>nDCG@10 |
-|:--|:--|:--|:--|:--|:----|
-|OpenAI/text-embedding-3-small|-|processing...|38.8|81.6|processing...|
-|OpenAI/text-embedding-3-large|-|processing...|processing...|processing...|processing...|
-||||||||||
-|[intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 0.6B | 89.2 | 55.4 | **87.6** | 29.8 |
-|[cl-nagoya/ruri-large](https://huggingface.co/cl-nagoya/ruri-large) | 0.3B | 78.7 | 62.4 | 85.0 | **37.5** |
-||||||||||
-|[intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 0.3B | 84.2| 47.2 | **85.3** | 25.4 |
-|[cl-nagoya/ruri-base](https://huggingface.co/cl-nagoya/ruri-base) | 0.1B | 74.3 | 58.1 | 84.6 | **35.3** |
-|[pkshatech/GLuCoSE-base-ja](https://huggingface.co/pkshatech/GLuCoSE-base-ja) | 0.1B | 53.3 | 30.8 | 68.6 | 25.2 |
-|**GLuCoSE v2**| 0.1B | **85.5** | **60.6** | **85.3** | 33.8 |
-
-Note: Results for OpenAI small embeddings in JQARA and JaCWIR are quoted from the [JQARA](https://huggingface.co/datasets/hotchpotch/JQaRA) and [JaCWIR](https://huggingface.co/datasets/hotchpotch/JCWIR).
-
+| :---: | :---: | :---: | :---: | :---: | :---: |
+| OpenAI/text-embedding-3-small | - | processing... | 38.8 | 81.6 | processing... |
+| OpenAI/text-embedding-3-large | - | processing... | processing... | processing... | processing... |
+|  |  |  |  |  |  |
+| [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 0.6B | 89.2 | 55.4 | **87.6** | 29.8 |
+| [cl-nagoya/ruri-large](https://huggingface.co/cl-nagoya/ruri-large) | 0.3B | 78.7 | 62.4 | 85.0 | **37.5** |
+|  |  |  |  |  |  |
+| [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 0.3B | 84.2 | 47.2 | **85.3** | 25.4 |
+| [cl-nagoya/ruri-base](https://huggingface.co/cl-nagoya/ruri-base) | 0.1B | 74.3 | 58.1 | 84.6 | **35.3** |
+| [pkshatech/GLuCoSE-base-ja](https://huggingface.co/pkshatech/GLuCoSE-base-ja) | 0.1B | 53.3 | 30.8 | 68.6 | 25.2 |
+| **GLuCoSE v2** | 0.1B | **85.5** | **60.6** | **85.3** | 33.8 |
+
+Note: Results for OpenAI small embeddings in JQARA and JaCWIR are quoted from the [JQARA](https://huggingface.co/datasets/hotchpotch/JQaRA) and [JaCWIR](https://huggingface.co/datasets/hotchpotch/JaCWIR).
 
 ### JMTEB
+
 Evaluated with [JMTEB](https://github.com/sbintuitions/JMTEB).
 
-|Model|Size|Avg.|Retrieval|STS|Classification|Reranking|Clustering|PairClassification|
-|:--|:--|:--|:--|:--|:--|:--|:--|:--|
-|OpenAI/text-embedding-3-small|-|70.86|66.39|79.46|73.06|92.92|51.06|62.27|
-|OpenAI/text-embedding-3-large|-|73.97|74.48|82.52|77.58|93.58|53.32|62.35|
-||||||||||
-|[intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large)|0.6B|71.65|70.98|79.70|72.89|92.96|51.24|62.15|
-|[cl-nagoya/ruri-large](https://huggingface.co/cl-nagoya/ruri-large)|0.3B|73.45|73.02|83.13|77.43|92.99|51.82|62.29|
-||||||||||
-|[intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base)|0.3B|70.12|68.21|79.84|69.30|**92.85**|48.26|62.26|
-|[cl-nagoya/ruri-base](https://huggingface.co/cl-nagoya/ruri-base) |0.1B|**72.95**|69.82|82.87|75.58|92.91|**54.16**|62.38|
-|[pkshatech/GLuCoSE-base-ja](https://huggingface.co/pkshatech/GLuCoSE-base-ja)|0.1B|70.44|59.02|78.71|**76.82**|91.90|49.78|**66.39**|
-|**GLuCoSE v2**|0.1B|72.39|**73.36**|**82.96**|74.21|93.01|48.65|62.37|
+| Model | Size | Avg. | Retrieval | STS | Classification | Reranking | Clustering | PairClassification |
+| :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
+| OpenAI/text-embedding-3-small | - | 69.18 | 66.39 | 79.46 | 73.06 | 92.92 | 51.06 | 62.27 |
+| OpenAI/text-embedding-3-large | - | 74.05 | 74.48 | 82.52 | 77.58 | 93.58 | 53.32 | 62.35 |
+|  |  |  |  |  |  |  |  |  |
+| [intfloat/multilingual-e5-large](https://huggingface.co/intfloat/multilingual-e5-large) | 0.6B | 70.90 | 70.98 | 79.70 | 72.89 | 92.96 | 51.24 | 62.15 |
+| [cl-nagoya/ruri-large](https://huggingface.co/cl-nagoya/ruri-large) | 0.3B | 73.31 | 73.02 | 83.13 | 77.43 | 92.99 | 51.82 | 62.29 |
+|  |  |  |  |  |  |  |  |  |
+| [intfloat/multilingual-e5-base](https://huggingface.co/intfloat/multilingual-e5-base) | 0.3B | 68.61 | 68.21 | 79.84 | 69.30 | **92.85** | 48.26 | 62.26 |
+| [cl-nagoya/ruri-base](https://huggingface.co/cl-nagoya/ruri-base) | 0.1B | 71.91 | 69.82 | 82.87 | 75.58 | 92.91 | **54.16** | 62.38 |
+| [pkshatech/GLuCoSE-base-ja](https://huggingface.co/pkshatech/GLuCoSE-base-ja) | 0.1B | 67.29 | 59.02 | 78.71 | **76.82** | 91.90 | 49.78 | **66.39** |
+| **GLuCoSE v2** | 0.1B | **72.23** | **73.36** | **82.96** | 74.21 | 93.01 | 48.65 | 62.37 |
 
 Note: Results for OpenAI embeddings and multilingual-e5 models are quoted from the [JMTEB leaderboard](https://github.com/sbintuitions/JMTEB/blob/main/leaderboard.md). Results for ruri are quoted from the [cl-nagoya/ruri-base model card](https://huggingface.co/cl-nagoya/ruri-base/blob/main/README.md).
 
-9/11 correction: Some values were initially micro-averaged; I've now standardized all metrics to macro-averaging for consistency.
-
 ## Authors
+
 Chihiro Yano, Mocho Go, Hideyuki Tachibana, Hiroto Takegawa, Yotaro Watanabe
 
 ## License
-This model is published under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0).
-
-<!--
-## Citation
-
-### BibTeX
-## Glossary
-
-*Clearly define terms in order to be accessible across audiences.*
--->
-
-<!--
-## Model Card Authors
-
-*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
--->
-
-<!--
-## Model Card Contact
 
-*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--->
+This model is published under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0).