Veyra-50M: Fast Multi-Domain Text Classification

Veyra-50M is a compact, encoder-only text classification model developed by Dl26. It is designed for fast English classification across intents, banking support requests, news topics, encyclopedia-style topics, emotion categories, sentiment labels, and broad question topics.

The model uses a BERT-style bidirectional Transformer encoder and the standard Hugging Face Transformers bert model type. It loads directly with AutoModelForSequenceClassification and does not require custom architecture files or trust_remote_code=True.

Veyra-50M is built for practical classification workflows where speed, simple deployment, and a wide label taxonomy matter. It can be used for routing user messages, classifying support requests, enriching metadata, triaging documents, filtering datasets, and building lightweight classifier services.

Why this model

• Compact encoder-only classifier
• 267-label multi-domain taxonomy
• Standard Hugging Face Transformers compatibility
• No custom Python architecture files
• Fast CPU and GPU batch inference
• Strong fit for routing, triage, metadata enrichment, and query classification
• Supports intent, topic, sentiment, emotion, and support-style labels in one model

Model details

Property	Value
Model name	Veyra-50M
Developer	Dl26
Model type	Encoder-only sequence classifier
Transformers model type	bert
Architecture	BertForSequenceClassification
Parameters	47,814,923
Hidden size	512
Layers	10
Attention heads	8
Intermediate size	2,048
Max positions	512
Number of labels	267
Training objective	Supervised single-label classification
License	Apache 2.0

Label taxonomy

Veyra-50M predicts one label from a namespaced taxonomy. The namespace indicates the task family or source label space.

Family	Labels	Purpose
clinc_oos	151	General assistant intent routing and out-of-scope detection
banking77	77	Banking and financial support intent classification
dbpedia_14	14	Broad entity and encyclopedia topic classification
yahoo_answers_topics	10	Broad question topic classification
emotion	6	Emotion classification
sst5	5	Fine-grained sentiment classification
ag_news	4	News topic classification

Example labels:

ag_news::Business
ag_news::Sci/Tech
banking77::activate_my_card
banking77::card_payment_not_recognised
clinc_oos::book_flight
clinc_oos::weather
dbpedia_14::Company
dbpedia_14::Film
emotion::joy
emotion::sadness
sst5::4
yahoo_answers_topics::Science & Mathematics

The full label map is stored in config.json as id2label and label2id.

Training data

Veyra-50M was trained on a multi-domain English classification mixture. The training setup combines short user utterances, customer-support style messages, news snippets, topic classification examples, question categories, emotion labels, and sentiment examples.

Source	Train	Validation	Test
clinc_oos	15,250	3,100	5,500
banking77	9,993	3,076	3,076
ag_news	120,000	7,600	7,600
dbpedia_14	220,000	8,000	8,000
yahoo_answers_topics	220,000	8,000	8,000
emotion	16,000	2,000	2,000
sst5	8,544	1,101	2,210

The labels are namespaced so that labels from different tasks remain distinct. For example, a banking support label and a general assistant intent label can both describe payments or account activity, but they remain separate targets.

Installation

pip install -U transformers torch accelerate

For CPU-only inference, accelerate is optional:

pip install -U transformers torch

Quick start

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-50M"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

text = "Can you help me reset my password?"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=64)

with torch.no_grad():
    logits = model(**inputs).logits

label_id = int(logits.argmax(dim=-1))
print(model.config.id2label[label_id])

Batch inference

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "Dl26/Veyra-50M"
device = "cuda" if torch.cuda.is_available() else "cpu"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id).to(device)
model.eval()

texts = [
    "Please freeze my card, I think it was stolen.",
    "The company announced a new processor for its laptops.",
    "I feel really happy about the result today.",
    "Who won the match last night?",
]

inputs = tokenizer(
    texts,
    return_tensors="pt",
    truncation=True,
    padding=True,
    max_length=64,
).to(device)

with torch.no_grad():
    logits = model(**inputs).logits

for text, label_id in zip(texts, logits.argmax(dim=-1).tolist()):
    print(model.config.id2label[label_id], "-", text)

Confidence scores

import torch

with torch.no_grad():
    logits = model(**inputs).logits
    scores = torch.softmax(logits, dim=-1)

top_scores, top_ids = scores.topk(k=3, dim=-1)

for row_scores, row_ids in zip(top_scores, top_ids):
    for score, label_id in zip(row_scores.tolist(), row_ids.tolist()):
        print(model.config.id2label[label_id], round(score, 4))

Confidence scores are useful for routing and fallback behavior. For production systems, calibrate thresholds on data from the target application.

Evaluation highlights

Evaluation	Result
Validation accuracy	83.15%
Validation macro-F1	85.28%
Test accuracy	80.65%
Test macro-F1	83.31%
Remote code required	No

Intended use

Veyra-50M is intended for:

• intent classification
• support request routing
• customer-service triage
• topic classification
• emotion and sentiment classification
• metadata enrichment
• dataset filtering
• search and recommendation labeling
• lightweight classification APIs
• encoder model research

Input formatting

The model works best with short to medium English text. For records with multiple fields, combine the fields into a single natural-language string before classification.

text = f"{title}. {description}"

Recommended defaults:

tokenizer(text, truncation=True, padding=True, max_length=64)

Use max_length=128 or higher when classifying longer documents, but expect slower inference.

Limitations

• Veyra-50M is a classifier, not a generative model.
• The model predicts within its fixed 267-label taxonomy.
• It is trained for English text.
• Very short, vague, adversarial, or out-of-domain inputs may be ambiguous.
• Confidence scores may need calibration for production use.
• Human review is recommended for high-impact decisions.

Citation

@misc{dl26_2026_veyra_50m,
  title        = {Veyra-50M: Fast Multi-Domain Text Classification},
  author       = {Dl26},
  year         = {2026},
  url          = {https://huggingface.co/Dl26/Veyra-50M}
}