QLTAN-1.0 (QualiText Analysis Network)

Model Description

QLTAN-1.0 is a fine-tuned DistilBERT model designed for multi-class text origin and quality classification. The model categorizes text into five distinct classes to help identify the provenance and quality characteristics of textual content.

The model is based on distilbert-base-uncased and was fine-tuned on the QualiText dataset, achieving strong performance in distinguishing between human-written, machine-generated, corrupted, marketing, and low-quality text.

Model Classes

Class ID	Label	Description
0	corrupted	Text with deliberate word deletions, swaps, duplications, or typo-like edits
1	human	Human-authored text from Wikipedia and 4chan
2	low_quality	Spam, email corpus examples, and text with removed punctuation and shuffled words
3	machine_generated	Text generated by Qwen3.8-Max, GLM-5.2, and Kimi-K3 models
4	marketing	Marketing-oriented content from FineWeb-Marketing

Intended Uses

The model is designed for:

• Text provenance analysis: Identifying whether text was human-written or machine-generated
• Data quality assessment: Detecting corrupted or low-quality text in datasets
• Content moderation: Filtering marketing content or spam from user-generated content
• Dataset cleaning: Automatically identifying problematic examples in training data
• Research: Studying text quality characteristics and synthetic data detection

Limitations

• Not for authorship verification: The model cannot definitively prove whether a specific person wrote text or used AI assistance
• Domain-specific performance: Performance may vary on text from domains not represented in the training data
• Language restriction: Model is trained on English text only
• Style bias: The machine-generated class may reflect the specific styles of the teacher models used in training data
• Adversarial vulnerability: The model may be fooled by sophisticated adversarial attacks or novel text generation methods

Training Data

The model was trained on the QualiText dataset, which contains 492,275 balanced examples across five classes (98,455 per class). The dataset was split with 90% for training and 10% for evaluation.

The source datasets include:

• Wikipedia (human and corrupted classes)
• 4chan political discussions (human class)
• Qwen3.8-Max, GLM-5.2, Kimi-K3 distillation corpus (machine-generated class)
• FineWeb-Marketing (marketing and low-quality classes)
• Spam/ham/phishing email corpus (low-quality class)

Training Procedure

Preprocessing

Text was split into sentence windows with:

• Maximum 6 sentences per window
• Maximum 3,000 characters per window
• Fallback of 256 words for text without clear sentence boundaries
• Maximum token length of 256

Hyperparameters

Parameter	Value
Learning rate	2e-05
Train batch size	32
Eval batch size	64
Gradient accumulation steps	1
Weight decay	0.01
Warmup ratio	0.1
Epochs	3
Mixed precision	BF16
Gradient checkpointing	No
Optimizer	AdamW (torch_fused)
LR scheduler	Linear
Seed	42

Training Results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
0.1058	1.0	54,138	0.1317	0.9560	0.9568
0.0797	2.0	108,276	0.1172	0.9621	0.9623
0.0435	3.0	162,414	0.1481	0.9628	0.9631

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
model_name = "akaruineko/qltan-1.0"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Example text
text = "The quick brown fox jumps over the lazy dog. This is a test sentence."

inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=256, return_token_type_ids=False)

with torch.no_grad():
    outputs = model(**inputs)
    predictions = torch.softmax(outputs.logits, dim=-1)

# Get predicted class
predicted_class = torch.argmax(predictions, dim=-1).item()
class_labels = {0: "corrupted", 1: "human", 2: "low_quality",
                3: "machine_generated", 4: "marketing"}
print(f"Predicted class: {class_labels[predicted_class]}")
print(f"Confidence: {predictions[0][predicted_class]:.4f}")

Environmental Impact

The model was trained on an A100 GPU for approximately 2 hours, resulting in roughly 162,414 training steps. The total computational cost is relatively modest due to the efficient DistilBERT architecture.

Framework Versions

• Transformers 5.15.0
• PyTorch 2.11.0+cu128
• Datasets 4.0.0
• Tokenizers 0.22.2