README.md

---
license: mit
language:
- en
library_name: transformers
pipeline_tag: text-classification
tags:
- phishing
- email-security
- bert
- text-classification
base_model: bert-base-uncased
widget:
- text: "Your account has been suspended. Verify now at http://secure-login.example.ru"
  example_title: Phishing example
- text: "Reminder: the team retro is tomorrow at 10am in the Oak meeting room."
  example_title: Legitimate example
---

# BERT Email Phishing Detector

Fine-tuned [`bert-base-uncased`](https://huggingface.co/bert-base-uncased) for
binary classification of email/message text as **phishing** or **legitimate**.

## Labels

| id | label       |
|----|-------------|
| 0  | legitimate  |
| 1  | phishing    |

## Intended use

- Research and education on phishing detection.
- A component / baseline in a larger email-security pipeline.

**Not intended** as a standalone production email security gateway. Always keep
a human in the loop for high-stakes decisions, and combine with URL reputation,
sender authentication (SPF/DKIM/DMARC) and other signals.

## How to use

```python
from transformers import pipeline

clf = pipeline("text-classification", model="<your-username>/bert-phishing-detector")
clf("Verify your account now at http://secure-login.example.ru")
# [{'label': 'phishing', 'score': 0.99}]
```

## Training

- **Base model:** `bert-base-uncased`
- **Objective:** binary sequence classification (cross-entropy)
- **Max sequence length:** 256 tokens
- **Optimizer / LR:** AdamW, 2e-5, weight decay 0.01, 6% warmup
- **Epochs:** 3 (best checkpoint by validation F1)

Training code: [`src/train.py`](src/train.py) in the project repository.

## Evaluation

Fine-tuned for 3 epochs on ~65.7k emails and evaluated on an 8,207-email
held-out test split:

| Metric    | Score  |
|-----------|--------|
| Accuracy  | 0.9948 |
| Precision | 0.9956 |
| Recall    | 0.9944 |
| F1        | 0.9950 |

> These scores are very high partly because the training data (the Kaggle
> phishing-email collection) is cleanly separable. Expect lower performance on
> noisier, real-world email — validate on your own data before relying on it.

## Limitations & biases

- **Distribution shift is the big one.** The training corpus is heavily
  preprocessed (lowercased, punctuation and stop-words removed) and the
  legitimate emails skew toward early-2000s corporate mail (Enron). The model
  scores ~99% on held-out data from this same distribution, but can misclassify
  raw, modern, or very short emails (e.g. a casual "Hi team, notes attached"
  message may be flagged as phishing). For best results, feed it text
  preprocessed the same way as training, and treat the headline metrics as
  in-distribution — not a promise of real-world accuracy.
- Trained primarily on **English** text; performance on other languages is
  untested.
- Phishing tactics evolve quickly; the model can go stale and should be
  periodically retrained on fresh data.
- May produce false positives on legitimate marketing emails that share
  surface features with phishing (urgency, links, "verify" language).
- The bundled `data/sample_emails.csv` is a tiny illustrative sample. Train on a
  large, representative corpus before relying on any metric.

## Data

Bring your own labeled corpus with a `text` column and an integer `label`
column (0 = legitimate, 1 = phishing). Commonly used public sources include the
Nazario phishing corpus, the Enron email dataset (legitimate mail), SpamAssassin
public corpus, and various Kaggle phishing-email datasets. Check each dataset's
license before redistribution.

## Ethical considerations

This model classifies text and can make mistakes. Do not use it to
automatically delete mail or take irreversible actions without human review.