README.md

---
language:
- vi
license: cc-by-nc-sa-4.0
library_name: transformers
pipeline_tag: feature-extraction
tags:
- vietnamese
- contrastive-learning
- sentence-embedding
- natural-language-inference
- xlm-roberta
- low-resource
- nlu
datasets:
- ViNLI
metrics:
- f1
- accuracy
---

# ViCLSR

**ViCLSR** (Vietnamese Contrastive Learning for Sentence Representations) is a supervised contrastive learning framework for Vietnamese Natural Language Understanding (NLU). The model leverages Natural Language Inference (NLI) datasets to learn high-quality sentence embeddings using entailment and contradiction relationships, targeting low-resource Vietnamese NLU settings.

- 📄 **Paper:** [arXiv:2603.21084](https://arxiv.org/abs/2603.21084)
- 🤗 **Model:** [huynhtin/ViCLSR](https://huggingface.co/huynhtin/ViCLSR)

---

## Model Details

| Field | Details |
|---|---|
| **Model name** | ViCLSR |
| **Base model** | XLM-RoBERTa-Large |
| **Language** | Vietnamese |
| **Task** | Sentence Embedding / NLU |
| **Training objective** | Supervised Contrastive Learning with NLI |
| **License** | CC BY-NC-SA 4.0 |
| **Paper** | arXiv:2603.21084 (March 2026) |

---

## Abstract

High-quality text representations are crucial for NLU, but low-resource languages like Vietnamese face challenges due to limited annotated data. We propose ViCLSR, a novel supervised contrastive learning framework that optimizes sentence embeddings for Vietnamese by leveraging existing NLI datasets. ViCLSR significantly outperforms strong baselines on five Vietnamese NLU benchmarks, demonstrating that supervised contrastive learning can effectively address resource limitations in low-resource NLU tasks.

---

## Performance

ViCLSR is evaluated on five Vietnamese NLU benchmarks spanning NLI, Fact Checking, Constructive Speech Detection, and Reading Comprehension (Table 4 in the paper).

### ViCLSR Results

| Dataset | Task | Metric | ViCLSR | vs. XLM-R Large | vs. PhoBERT Large |
|---|---|---|---|---|---|
| ViNLI | Natural Language Inference | F1 | **82.84** | ↑1.53 | ↑6.97 |
| ViWikiFC | Fact Checking | F1 | **86.57** | ↑1.42 | ↑4.97 |
| ViFactCheck | Fact Checking | F1 | **88.78** | ↑0.76 | ↑9.02 |
| UIT-ViCTSD | Constructive Speech Detection | F1 | **82.22** | ↑2.78 | ↑5.36 |
| ViMMRC2.0 | Reading Comprehension | Acc | **59.06** | ↑1.54 | ↑4.33 |

> Full results and analysis are available in the [paper](https://arxiv.org/abs/2603.21084).

---

## Intended Uses

ViCLSR is designed for Vietnamese NLU research and can be applied to:

- ✅ Sentence embedding
- ✅ Semantic similarity
- ✅ Natural Language Inference (NLI)
- ✅ Information retrieval
- ✅ Fact checking
- ✅ Sentiment analysis
- ✅ Vietnamese NLU tasks in general

### Out-of-Scope Uses

- ❌ Non-Vietnamese languages (model is optimized for Vietnamese)
- ❌ Commercial use (CC BY-NC-SA 4.0 license)

---

## Usage

### Installation

```bash
pip install transformers torch
```

### Sentence Embedding

```python
from transformers import AutoTokenizer, AutoModel
import torch
import torch.nn.functional as F

model_name = "huynhtin/ViCLSR"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
model.eval()

def get_embedding(text):
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=256)
    with torch.no_grad():
        outputs = model(**inputs)
    # Use [CLS] token representation
    embedding = outputs.last_hidden_state[:, 0]
    return embedding

text = "Trí tuệ nhân tạo đang phát triển rất nhanh."
embedding = get_embedding(text)
print(f"Embedding shape: {embedding.shape}")  # (1, 1024)
```

### Semantic Similarity

```python
from transformers import AutoTokenizer, AutoModel
import torch
import torch.nn.functional as F

model_name = "huynhtin/ViCLSR"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
model.eval()

def get_embedding(text):
    inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=256)
    with torch.no_grad():
        outputs = model(**inputs)
    embedding = outputs.last_hidden_state[:, 0]
    return F.normalize(embedding, dim=-1)

sentence1 = "Hà Nội là thủ đô của Việt Nam."
sentence2 = "Thành phố Hà Nội là thủ đô nước Việt Nam."
sentence3 = "Bóng đá là môn thể thao phổ biến nhất thế giới."

emb1 = get_embedding(sentence1)
emb2 = get_embedding(sentence2)
emb3 = get_embedding(sentence3)

sim_12 = (emb1 * emb2).sum().item()
sim_13 = (emb1 * emb3).sum().item()

print(f"Similarity (sentence1 vs sentence2): {sim_12:.4f}")  # High
print(f"Similarity (sentence1 vs sentence3): {sim_13:.4f}")  # Low
```

---

## Training Details

- **Base model:** XLM-RoBERTa-Large
- **Training framework:** Supervised Contrastive Learning
- **Training data:** Vietnamese NLI datasets (entailment/contradiction pairs)
- **Objective:** Contrastive loss using positive (entailment) and negative (contradiction) pairs
- **Language:** Vietnamese

---

## Limitations

- Optimized specifically for **Vietnamese** — performance may degrade significantly on other languages
- Performance depends on the quality and domain of input text
- Best suited for **research purposes** under CC BY-NC-SA 4.0

---

## Citation

If you use ViCLSR in your research, please cite:

```bibtex
@article{huynh2026viclsr,
  title={ViCLSR: A Supervised Contrastive Learning Framework with Natural Language Inference for Natural Language Understanding Tasks},
  author={Huynh, Tin Van and Nguyen, Kiet Van and Nguyen, Ngan Luu-Thuy},
  journal={arXiv preprint arXiv:2603.21084},
  year={2026}
}
```

---

## Authors

- **Tin Van Huynh** 
- **Kiet Van Nguyen** 
- **Ngan Luu-Thuy Nguyen**