valezion/url-phishing-detector

Phishing URL detector

Judges a link from the string alone: no network call, no page fetch, no external service. It can answer before anyone clicks — inside a mail filter, a form, or a CMS field.

Trained 23 September 2026. Scikit-learn 1.7.2, CPU only.

• Code, data collectors and the full story: github.com/valezion7/url-phishing-detector
• Live demo: phishing.studiobeezy.com
• The part worth reading: DIARY.md — eight times this model learned something that had nothing to do with phishing, and how each was caught

What it is

A StackingClassifier over two base models:

1. Logistic regression over character n-grams (3–5, char_wb) of the whole URL.
2. HistGradientBoosting over 62 computed features: lengths, subdomain count, digits, entropy, a known brand appearing in the hostname under a domain that is not its own, shared-hosting providers, login-page vocabulary in five languages, and two measures of how pronounceable the domain name is (dictionary-word coverage and character-bigram plausibility).

Deliberately excluded: the http/https scheme, the trailing slash, the www. prefix. In any dataset assembled from public feeds, those encode which list the URL came from, not whether it is dangerous. Leaving them in is the single easiest way to get a great benchmark number and a useless model.

Results

Trained on 260,000 URLs (half live phishing, half benign web) across about 195,000 domains.

test	ROC-AUC
random split	0.989
split by domain (no domain shared between train and test)	0.987
independent holdout (today's verified phishing vs links people posted today)	0.948

On the independent holdout:

good links blocked	fresh phishing caught	threshold	URLhaus malware caught
0.50%	45.6%	0.984	91.1%
0.90%	53.4%	0.977	92.8%
1.99%	62.7%	0.958	94.3%
4.89%	78.2%	0.844	96.3%
9.86%	86.0%	0.542	97.3%

The holdout has 2,210 benign links (Hacker News, lobste.rs, same day) and 193 verified phishing URLs (OpenPhish, same day), with no domain in common with training. With 193 positives the 53.4% carries a 95% bootstrap interval of 46.6–60.6%: an honest estimate, not a precise measurement.

The URLhaus column is a transfer test: 13,427 malware URLs, a different threat never seen in training.

How to use it

import sys, json, joblib
from huggingface_hub import snapshot_download

base = snapshot_download("valezion/url-phishing-detector")
sys.path.insert(0, base)
import features as F
F.RAW, F.DERIVATI = f"{base}/data/raw", f"{base}/vocabolario"   # uses the bundled vocabulary

model = joblib.load(f"{base}/model.joblib")
th = json.load(open(f"{base}/thresholds.json"))

urls = ["https://www.example.com/", "http://paypal.com.secure-login.verify.tk/webscr"]
scores = model.predict_proba(F.featurize(urls))[:, 1]

thresholds.json carries two operating points measured on the internal by-domain test: threshold_fpr1 (≈1% false positives) and threshold_fpr01 (≈0.1%). Pick yours from the holdout table above according to what an error costs you in each direction; there is no universally correct threshold.

Scores are not calibrated probabilities: they are heavily concentrated near 0 and 1. Use them for ranking and thresholding, not as "probability of phishing".

Intended use, and what it is not

Intended: triage and ranking of large URL lists, a first-stage signal inside a larger system, research and teaching about dataset bias.

Not a security product, and not a substitute for one. It is blind to phishing hosted inside legitimate compromised sites, where the string carries no evidence at all. A low score is never a guarantee that a link is safe. Do not use it as the sole gate on anything that matters.

Limits

• Compromised legitimate sites: the ceiling of the problem, not of the model.
• English dictionary: the two readability measures use an English word list, so domains in other languages start at a disadvantage. The training data is multilingual (English, Italian, German, French, Spanish); the dictionary is not yet.
• No reputation signals: no domain age, certificate or IP reputation. Those live outside a string-only model and would raise the result substantially.
• A known false positive: accounts.google.com/signin still scores as phishing. The fix is more big-provider login pages among the benign examples.
• It ages. Phishing moves; the feeds change daily. A model frozen in September 2026 decays. The repository contains everything needed to retrain.
• Fairness note: the model reads brand names and language cues. It has not been audited for systematic bias against particular languages, regions or hosting providers beyond the limits listed here.

Training data

No pre-packaged dataset. The most-cited academic dataset for this task (PhiUSIIL) is unusable: every one of its legitimate URLs is https://, ≤58 characters, without a path, while its phishing URLs reach 6,097 characters. A model trained on it learns length.

Phishing from Phishing.Database (ACTIVE list). Benign from Wikipedia external links (English plus Italian, German, French and Spanish, and queried TLD by TLD), sitemaps of Tranco-listed sites, Tranco homepages sampled across the whole ranking, and 3,941 real login pages verified with HTTP 200. Held out entirely: OpenPhish, Hacker News, lobste.rs, URLhaus.

The collected data is not redistributed here — several of those feeds do not allow it. The collectors are in the GitHub repository, so the dataset can be rebuilt from the live sources.

Bundled in this repository: features.py (the feature extraction, identical to training) and vocabolario/ — a compact derived vocabulary (3,714 brand labels, 6,484 reference domains, an English word list) so the model runs without rebuilding the dataset.

Citation of sources

Tranco (Le Pochat et al., NDSS 2019), URLhaus (abuse.ch), OpenPhish, Phishing.Database (mitchellkrogza), Wikipedia, the dwyl/english-words list.

Code MIT. The model weights are released under the same terms; the underlying feeds keep their own licences.