ViLegalQwen2.5-1.5B-Base

ViLegalQwen2.5-1.5B-Base is a decoder-only language model for Vietnamese legal text understanding, part of the ViLegalLM suite. It is continually pretrained from Qwen2.5-1.5B on a newly curated 16GB Vietnamese legal corpus. ViLegalQwen2.5-1.5B-Base achieves state-of-the-art results among 1.5B-scale models across Vietnamese legal downstream tasks including Question Answering, Natural Language Inference, and Syllogism Reasoning.

Paper: ViLegalLM: Language Models for Vietnamese Legal Text — ACL 2026

Resources: GitHub | ViLegalBERT | ViLegalQwen3-1.7B-Base

---

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("ntphuc149/ViLegalQwen2.5-1.5B-Base")
model = AutoModelForCausalLM.from_pretrained("ntphuc149/ViLegalQwen2.5-1.5B-Base")

Note: This is a base (pretrained) model, not an instruction-tuned model. We do not recommend using base language models for conversations. Instead, you can apply post-training, e.g., SFT, RLHF, continued pretraining, etc., on this model.

---

Model Summary

Summary for ViLegalQwen2.5-1.5B-Base checkpoint (click to expand)

Attribute	Value
Architecture	Qwen2 (decoder-only, causal LM)
Parameters	1.54B
Base model	Qwen2.5-1.5B
Max sequence length	2048 tokens
Tokenizer	Qwen2 tokenizer
Training objective	Causal Language Modeling (CLM)
Training domain	Vietnamese legal text
Precision	BF16

---

Evaluation Results

ViLegalQwen2.5-1.5B-Base achieves state-of-the-art results among 1.5B-scale models across all evaluated Vietnamese legal benchmarks. Bold = best in the 1.5B parameter group. Italic = closed-source model scores.

Question Answering (click to expand)

True/False — ALQAC-TF

Model	Pre	Rec	F1
Qwen2-1.5B	85.05	86.84	85.94
Qwen2.5-1.5B	74.47	92.11	83.34
ViLegalQwen2.5-1.5B-Base	87.31	90.53	88.89
gpt-4o-mini (0-shot)	89.86	97.89	93.70

Multiple Choice — ALQAC-MCQ & VLSP-MCQ-LK

Model	Pre_mac	Rec_mac	F1_mac
ALQAC-MCQ
Qwen2-1.5B	82.19	81.05	81.42
Qwen2.5-1.5B	84.80	84.05	84.37
ViLegalQwen2.5-1.5B-Base	85.66	84.53	84.96
gpt-4o-mini (0-shot)	90.83	91.58	91.15
VLSP-MCQ-LK
Qwen2-1.5B	68.02	53.62	58.15
Qwen2.5-1.5B	65.05	52.34	56.54
ViLegalQwen2.5-1.5B-Base	65.24	54.54	58.39
gpt-4o-mini (0-shot)	69.05	51.16	58.17

Abstractive QA — ViBidLQA-AQA

Model	ROUGE-L	BLEU-4	BS-F1
Qwen2-1.5B	72.44	49.00	89.68
Qwen2.5-1.5B	73.02	49.12	89.74
ViLegalQwen2.5-1.5B-Base	73.45	49.90	89.91
gpt-4o-mini (0-shot)	67.06	40.46	86.85

Natural Language Inference (click to expand)

NLI — VLSP-NLI

Model	Precision	Recall	F1
Qwen2-1.5B	92.86	86.67	89.66
Qwen2.5-1.5B	100.00	80.00	88.89
ViLegalQwen2.5-1.5B-Base	84.90	100.00	91.84
gpt-4o-mini (0-shot)	100.00	86.67	92.86

Syllogism Reasoning (click to expand)

Syllogism Reasoning — VLSP-Syllogism

Model	BS-F1	LLM-Judge
Qwen2-1.5B	76.19	0.2639
Qwen2.5-1.5B	76.89	0.2656
ViLegalQwen2.5-1.5B-Base	76.63	0.2674
gpt-4o-mini (0-shot)	78.63	0.5069

---

Also in ViLegalLM

Model	Architecture	Params	Context
ViLegalBERT	Encoder-only	135M	256
ViLegalQwen2.5-1.5B-Base (this model)	Decoder-only	1.54B	2,048
ViLegalQwen3-1.7B-Base	Decoder-only	1.72B	4,096

---

Limitations and Biases

• Domain scope: Trained exclusively on Vietnamese legal texts; may not generalize to other legal systems or jurisdictions.
• Base model only: Not instruction-tuned; outputs may be incomplete or incoherent without task-specific fine-tuning.
• Temporal bias: Legal corpus reflects Vietnamese law as of the collection date; model outputs may not reflect recent legislative changes.
• Inherited biases: May reflect biases present in the source legal corpora, including regional variations in legal practice and domain coverage imbalances.
• Not a legal authority: Model outputs should never be used as definitive legal interpretations without expert validation.

---

Intended Use

Intended for:

• Further fine-tuning on Vietnamese legal downstream tasks (QA, NLI, reasoning)
• Vietnamese legal text generation and completion
• Research on Vietnamese legal NLP and continual pretraining

Not intended for:

• Direct conversational or instruction-following use without fine-tuning
• Replacing professional legal counsel or human judgment in legal decision-making
• Providing legal advice without expert validation
• Legal systems outside Vietnam without careful domain adaptation

---

Citation

If you use ViLegalQwen2.5-1.5B-Base, please cite our paper:

<!-- ViLegalLM citation — available soon -->

---

License

Apache-2.0

This model is derived from Qwen2.5-1.5B which is licensed under Apache-2.0. ViLegalQwen2.5-1.5B-Base is released under the same Apache-2.0 license.