vietnamese-prompt-injection

Prompt-injection classifier for Vietnamese (with English retained from the base model), fine-tuned from protectai/deberta-v3-base-prompt-injection-v2. Labels: SAFE / INJECTION.

General-domain revision: Vietnamese prompt-injection detection for assistant traffic without domain-specific tuning. Domain-tuned variants live on other branches of this repo.

Held-out evaluation

FP = false-positive rate on benign cells, TP = true-positive rate on injection cells. Threshold 0.5.

Evaluation cell	n	FP	TP
VN benign — domain-specific hard negatives	300	0.0%	—
VN benign — real user queries	200	0.0%	—
VN benign — general conversation	100	0.0%	—
VN injection — camouflaged	200	—	100.0%
EN injection	120	—	95.8%
EN benign	150	8.7%	—
VN injection — standalone bare imperatives	80	—	100.0%
VN benign — standalone bare imperatives	80	0.0%	—
VN injection — diacritic-stripped	70	—	100.0%
VN benign — diacritic-stripped	70	1.4%	—

Training data (statistics only)

• Training pool: 15,111 samples — 8,346 SAFE / 6,765 INJECTION, 10 scenario cells.
• Held-out evaluation: 1,370 samples, disjoint from training (near-duplicate filtered, 5-gram Jaccard >= 0.85).
• Mix of synthetic template-generated Vietnamese (formal, informal/abbreviated, diacritic-stripped layers), real anonymized user queries, and public prompt-injection / benign corpora.

Usage

from transformers import pipeline

pipe = pipeline(
    "text-classification",
    model="dangvansam/vietnamese-prompt-injection",
)
pipe("Cho tôi danh sách các quy trình bạn nắm được")
# -> [{'label': 'SAFE', 'score': ...}]

With llm-guard:

from llm_guard.input_scanners import PromptInjection
from llm_guard.model import Model

scanner = PromptInjection(
    model=Model(
        path="dangvansam/vietnamese-prompt-injection",
        pipeline_kwargs={"return_token_type_ids": False, "max_length": 512, "truncation": True},
    ),
    threshold=0.5,
)