README.md

metadata

language:
  - vi
license: cc-by-nc-sa-4.0
library_name: transformers
pipeline_tag: feature-extraction
tags:
  - vietnamese
  - contrastive-learning
  - sentence-embedding
  - natural-language-inference
  - xlm-roberta
  - low-resource
  - nlu
datasets:
  - ViNLI
metrics:
  - f1
  - accuracy

ViCLSR

ViCLSR (Vietnamese Contrastive Learning for Sentence Representations) is a supervised contrastive learning framework for Vietnamese Natural Language Understanding (NLU). The model leverages Natural Language Inference (NLI) datasets to learn high-quality sentence embeddings using entailment and contradiction relationships, targeting low-resource Vietnamese NLU settings.

• 📄 Paper: arXiv:2603.21084
• 🤗 Model: huynhtin/ViCLSR

---

Model Details

Field	Details
Model name	ViCLSR
Base model	XLM-RoBERTa-Large
Language	Vietnamese
Task	Sentence Embedding / NLU
Training objective	Supervised Contrastive Learning with NLI
License	CC BY-NC-SA 4.0
Paper	arXiv:2603.21084 (March 2026)

---

Abstract

High-quality text representations are crucial for NLU, but low-resource languages like Vietnamese face challenges due to limited annotated data. We propose ViCLSR, a novel supervised contrastive learning framework that optimizes sentence embeddings for Vietnamese by leveraging existing NLI datasets. ViCLSR significantly outperforms strong baselines on five Vietnamese NLU benchmarks, demonstrating that supervised contrastive learning can effectively address resource limitations in low-resource NLU tasks.

---

Performance

ViCLSR is evaluated on five Vietnamese NLU benchmarks spanning NLI, Fact Checking, Constructive Speech Detection, and Reading Comprehension (Table 4 in the paper).

ViCLSR Results

Dataset	Task	Metric	ViCLSR	vs. XLM-R Large	vs. PhoBERT Large
ViNLI	Natural Language Inference	F1	82.84	↑1.53	↑6.97
ViWikiFC	Fact Checking	F1	86.57	↑1.42	↑4.97
ViFactCheck	Fact Checking	F1	88.78	↑0.76	↑9.02
UIT-ViCTSD	Constructive Speech Detection	F1	82.22	↑2.78	↑5.36
ViMMRC2.0	Reading Comprehension	Acc	59.06	↑1.54	↑4.33

Full results and analysis are available in the paper.

---

Intended Uses

ViCLSR is designed for Vietnamese NLU research and can be applied to:

• ✅ Sentence embedding
• ✅ Semantic similarity
• ✅ Natural Language Inference (NLI)
• ✅ Information retrieval
• ✅ Fact checking
• ✅ Sentiment analysis
• ✅ Vietnamese NLU tasks in general

Out-of-Scope Uses

• ❌ Non-Vietnamese languages (model is optimized for Vietnamese)
• ❌ Commercial use (CC BY-NC-SA 4.0 license)

---

Usage

Installation

pip install transformers torch

Sentence Embedding

from transformers import AutoTokenizer, AutoModel
import torch

model_name = "huynhtin/ViCLSR"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)

text = "Trí tuệ nhân tạo đang phát triển rất nhanh."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

# CLS sentence embedding
embedding = outputs.last_hidden_state[:, 0]

print(embedding.shape)  # torch.Size([1, 1024])

Semantic Similarity

from transformers import AutoTokenizer, AutoModel
import torch
import torch.nn.functional as F

model_name = "huynhtin/ViCLSR"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
model.eval()

def get_embedding(text):
    inputs = tokenizer(
        text,
        return_tensors="pt",
        padding=True,
        truncation=True,
        max_length=256
    )

    with torch.no_grad():
        outputs = model(**inputs)

    embedding = outputs.last_hidden_state[:, 0]
    return F.normalize(embedding, dim=-1)

sentence1 = "Trí tuệ nhân tạo đang thay đổi cách con người làm việc."
sentence2 = "AI đang làm thay đổi phương thức làm việc của con người."
sentence3 = "Hôm nay thời tiết ở Thành phố Hồ Chí Minh rất nóng."

emb1 = get_embedding(sentence1)
emb2 = get_embedding(sentence2)
emb3 = get_embedding(sentence3)

sim_12 = (emb1 * emb2).sum().item()
sim_13 = (emb1 * emb3).sum().item()

print(f"Similarity (sentence1 vs sentence2): {sim_12:.4f}")
print(f"Similarity (sentence1 vs sentence3): {sim_13:.4f}")

---

Training Details

• Base model: XLM-RoBERTa-Large
• Training framework: Supervised Contrastive Learning
• Training data: Vietnamese NLI datasets (entailment/contradiction pairs)
• Objective: Contrastive loss using positive (entailment) and negative (contradiction) pairs
• Language: Vietnamese

---

Limitations

• Optimized specifically for Vietnamese — performance may degrade significantly on other languages
• Performance depends on the quality and domain of input text
• Best suited for research purposes under CC BY-NC-SA 4.0

---

Citation

If you use ViCLSR in your research, please cite:

@article{huynh2026viclsr,
  title={ViCLSR: A Supervised Contrastive Learning Framework with Natural Language Inference for Natural Language Understanding Tasks},
  author={Huynh, Tin Van and Nguyen, Kiet Van and Nguyen, Ngan Luu-Thuy},
  journal={arXiv preprint arXiv:2603.21084},
  year={2026}
}

---