README.md

---
language:
- ko
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
pipeline_tag: sentence-similarity
library_name: sentence-transformers
base_model:
- klue/roberta-large
---

# Frony Embed V2 (medium)
This is an efficient embedding model designed specifically for the Korean language.
It has been trained on a diverse set of data sources, including AI 허브, to ensure robust performance in a wide range of retrieval tasks.
The model demonstrates strong retrieval capabilities across:<br>

* Korean–Korean
* Korean–English
* English–Korean

To support resource-constrained environments, the model also provides compatibility with Matryoshka Embeddings, enabling retrieval even at reduced dimensions **(e.g., half of the original size)** without significant performance loss.
All training and data preprocessing were performed on **a single GPU (46VRAM)**, showcasing not only the model’s effectiveness but also its efficiency.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base Model:** klue/roberta-large
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 1024 / 512 dimensions
- **Similarity Function:** Cosine Similarity
- **Languages:** ko, en
- **License:** apache-2.0

### Datasets
This model is trained from many sources data including **AI 허브**.<br>
Total trained query and document pair is 500,000.<br>

### Training Details
The overall training process was conducted with reference to snowflake-arctic-2.0.<br>
**In V2, a three-stage training process was introduced as a key component of the overall learning strategy.**<br>
The training process consisted of three stages: Adaptation-training, Pre-training, and Post-training.

* In the adaptation-training stage, we observed through preliminary experiments that multi-vector retrieval consistently outperformed standard dense retrieval. To reflect this, we first trained the model using a multi-vector retrieval objective.
* In the pre-training stage, we introduced knowledge distillation, **where the multi-vector retrieval loss was distilled into the dense retrieval loss**. This allowed the model to capture fine-grained token-level similarity signals while being trained with in-batch negatives.
* In the post-training stage, we utilized the multilingual-e5-large model to mine hard negatives—specifically, the top 4 samples with a similarity score below a 99% threshold—and fine-tuned the model further using these examples.

Given the increasing prevalence of LLM-generated content, we also converted existing data into Markdown-style passages to improve retrieval performance on such formats.<br>
The types of data augmentation applied are as follows:

| Augmentation* | Description |
-----------|-----------|
| Pair concatenation | Multi-query & Multi-passage |
| Language transfer | Korean to English on query & passage |
| Style transfer | Plain sentences to Markdown description |
**Augmentation was carried out using the Gemma-3-12B*

### Evaluation
The evaluation consists of five dataset groups, and the results in the table represent the average retrieval performance across these five groups.
Three groups are subsets extracted from AI 허브 datasets.
One group is based on a specific sports regulation PDF, for which synthetic query and **markdown-style passage** pairs were generated using GPT-4o-mini.
The final group is a concatenation of all four aforementioned groups, providing a comprehensive mixed set.<br>
The following table presents the average retrieval performance across five dataset groups.

| Architecture                                           | Open/Closed | Accuracy@1 | Accuracy@3 | Accuracy@5 | Accuracy@10 |
|--------------------------------------------------------|-----------|-----------|-----------|-----------|------------|
| upstage-large                                          | Closed | 0.6323    | 0.8522    | 0.9068    | 0.9459     |
| dragonkue/snowflake-arctic-embed-l-v2.0-ko             | Open   | 0.6612    | 0.8396    | 0.8931    | 0.9390     |
| **FronyAI/frony-embed-medium-ko-v2**                       | Open   | **0.6805**    | **0.8375**    | 0.8819    | 0.9206     |
| FronyAI/frony-embed-medium-arctic-ko-v2.5              | Open   | 0.6942    | 0.8361    | 0.8807    | 0.9197     |
| FronyAI/frony-embed-medium-arctic-ko-v2.5 (half dim)   | Open   | 0.6778    | 0.8277    | 0.8726    | 0.9129     |
| **FronyAI/frony-embed-medium-ko-v2 (half dim)**            | Open   | **0.6722**    | **0.8274**    | 0.8712    | 0.9157     |
| nlpai-lab/KURE-v1                                      | Open   | 0.6434    | 0.8240    | 0.8788    | 0.9285     |
| FronyAI/frony-embed-medium-ko-v1                       | Open   | 0.6649    | 0.8040    | 0.8458    | 0.8876     |
| FronyAI/frony-embed-medium-ko-v1 (half dim)            | Open   | 0.6520    | 0.7923    | 0.8361    | 0.8796     |
| BAAI/bge-m3                                            | Open   | 0.5849    | 0.7763    | 0.8420    | 0.8985     |
| intfloat/multilingual-e5-large                         | Open   | 0.5764    | 0.7630    | 0.8267    | 0.8891     |
| Snowflake/snowflake-arctic-embed-l-v2.0                | Open   | 0.5726    | 0.7591    | 0.8232    | 0.8917     |
| jinaai/jina-embeddings-v3                              | Open   | 0.5270    | 0.7242    | 0.7953    | 0.8644     |
| openai-text-embedding-3-large                          | Closed | 0.4903    | 0.6621    | 0.7316    | 0.8149     |

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("FronyAI/frony-embed-medium-ko-v2")

# Run inference
# '<Q>' is special token for query.
queries = [
    '<Q>안녕하세요',
]
embeddings = model.encode(queries)

# '<P>' is special token for passage.
passages = [
    '<P>반갑습니다',
]
embeddings = model.encode(passages)

# Matryoshka Embeddings (half of the original dimension)
# '<Q>' is special token for query.
queries = [
    '<Q>안녕하세요',
]
embeddings = model.encode(queries, normalize_embeddings=False, convert_to_tensor=True)[:, :512]
embeddings = F.normalize(embeddings, p=2, dim=-1)
```

## Contact
Feel free to open an issue or pull request if you have any questions or suggestions about this project.
You also can email (flash659@gmail.com).