ThomasTschinkel/fake-news-detector

Fake News Detector

A high-accuracy binary classifier that identifies news articles as REAL or FAKE, built on fine-tuned RoBERTa-Large with a custom dual-pooling classification head.

Metric	Score
Accuracy	99.81%
F1 (macro)	99.81%
Precision	99.79%
Recall	99.84%
ROC-AUC	99.95%

Try It Now

Launch the interactive demo: https://huggingface.co/spaces/ThomasTschinkel/fake-news-detector-demo

Check out the GitHub repo: https://github.com/thomastschinkel/fake-news-detector

Quick Start

Pipeline (recommended)

from transformers import pipeline

detector = pipeline(
    "text-classification",
    model="ThomasTschinkel/fake-news-detector",
    trust_remote_code=True,
)

results = detector([
    "NASA confirms water discovery on Mars.",
    "SHOCKING: 5G towers cause mind control!!!",
])

print(results)

Manual inference

Use trust_remote_code=True because this model uses a custom architecture.

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

tokenizer = AutoTokenizer.from_pretrained("ThomasTschinkel/fake-news-detector")
model = AutoModelForSequenceClassification.from_pretrained(
    "ThomasTschinkel/fake-news-detector",
    trust_remote_code=True,
)

text = "Your news article text here..."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=1)

print(f"REAL: {probs[0][0]:.2%} | FAKE: {probs[0][1]:.2%}")

Architecture

Input text
    |
    v
RoBERTa-Large (24 layers, 1024 hidden dim)
    |
    +---- [CLS] token (1024-d)
    |
    +---- Mean pooling (1024-d)
    |
    +---- Concatenate -> 2048-d
              |
              v
     Linear(2048 -> 512)
     LayerNorm + GELU + Dropout(0.3)
              |
              v
     Linear(512 -> 128)
     GELU + Dropout(0.2)
              |
              v
     Linear(128 -> 2)
              |
              v
        REAL / FAKE

Why dual pooling? The CLS token captures a global summary, while mean pooling preserves information across all tokens.

Training Details

• Base model: roberta-large
• Dataset: https://www.kaggle.com/datasets/jillanisofttech/fake-or-real-news
• Split: 80/20 stratified train/validation
• Max sequence length: 512
• Mixed precision: FP16 on CUDA

Limitations

• English only
• 512 token truncation for long documents
• Single-dataset training can reduce out-of-domain generalization
• This model is a screening tool, not a factual truth oracle

Links

• Space: https://huggingface.co/spaces/ThomasTschinkel/fake-news-detector-demo
• Dataset: https://www.kaggle.com/datasets/jillanisofttech/fake-or-real-news

Citation

@misc{tschinkel2025fakenews,
  title  = {Fake News Detector: Fine-tuned RoBERTa-Large},
  author = {Thomas Tschinkel},
  year   = {2025},
  url    = {https://huggingface.co/ThomasTschinkel/fake-news-detector}
}