Podric/prowl-secret-encoder

Prowl secret encoder

A multilingual text classifier that scores whether a fragment of text (a line of code, a config value, a Jira comment, a log line, a chat message) contains a leaked credential. It is stage 3 of Prowl, a high-precision secret scanner, and handles the free-form and multilingual tail that regex and the linear model miss.

This model is not a standalone scanner. It is one stage of an ensemble; on its own it is a recall booster, not a precision oracle. To scan a repo, use Prowl.

What it does

• Input: a short text span (≤512 tokens; the tool feeds ~128).
• Output: two logits → softmax(...)[1] is P(text contains a secret).
• Decision: fires at P ≥ 0.90, a threshold calibrated on a held-out validation split to precision ≥ 0.95 (value-disjoint from the benchmark, no leakage).

Role in the ensemble

Prowl combines three stages by union: a Go regex/checksum/entropy cascade, a char+word TF-IDF logistic regression, and this encoder. Adding the encoder to the other two, measured on ProwlBench (3,843 leakage-safe cases):

Configuration	Precision	Recall	F1
cascade ∪ LR	0.974	0.848	0.909
+ this encoder	0.971	0.970	0.970

The encoder lifts recall by 0.12 at a 0.003 precision cost, and reaches recall 1.00 on German, French, Spanish, and Russian prose passwords - the free-form, multilingual tail that regex and the linear model miss.

Standalone recall at a fixed precision of 0.97, by channel:

code	Jira	Confluence	log
0.984	0.996	0.998	0.947

Usage

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tok = AutoTokenizer.from_pretrained("Podric/prowl-secret-encoder")
model = AutoModelForSequenceClassification.from_pretrained("Podric/prowl-secret-encoder").eval()

def secret_score(text: str) -> float:
    enc = tok(text, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        return torch.softmax(model(**enc).logits, -1)[0, 1].item()

secret_score('DB_PASSWORD = "tR4!nf0rce-2026-prod"')   # high
secret_score("the deployment finished without errors")  # low
# fire at >= 0.90

Example output

secret_score(text) on a few inputs (the model fires at P ≥ 0.90):

P(secret)	fires	input
1.000	yes	DB_PASSWORD = "tR4!nf0rce-2026-prod"
1.000	yes	Ihr neues Passwort lautet F4QPE91sc6iN ... (German prose)
1.000	yes	Пароль от прод-сервера: Zx9!kLmN2k24qP ... (Russian prose)
0.001	no	token = os.environ["SERVICE_TOKEN"] (env reference)
0.002	no	Развёртывание завершилось без ошибок ... (Russian log)
0.000	no	def calculate_total(items): return sum(...) (benign code)
0.000	no	API_KEY=your_api_key_here (placeholder)

It fires on real passwords, including free-form prose in German and Russian that has no fixed prefix for a regex to anchor, and stays near zero on benign code, logs, env-var references, and placeholders.

Training

• Base: distilbert-base-multilingual-cased (104 languages, 6 layers, 134M params).
• Objective: binary sequence classification (secret-bearing / not).
• Data: the Prowl secrets corpus, 503k labeled records across code, tickets, logs, and prose. Real-secret sources (CredData, HF PII sets) are held out by origin so the validation split is not a memorization check. Positives are oversampled and real-origin records up-weighted.
• Calibration: the operating threshold is fixed post-hoc on validation to a precision target, not learned, so it transfers to the ensemble's union rule.

Limitations

• Binary, not typed. It says secret / not secret; the secret type (AWS vs Stripe vs ...) comes from Prowl's cascade.
• A stage, not a scanner. High recall comes at a precision that only makes sense inside the ensemble, where the cascade supplies structured-token precision. Do not deploy it alone as a gate.
• Span-level, not token-level. It flags a span; Prowl localizes the exact value.
• Distilled size. Chosen for CPU-friendly latency over a larger encoder; the few remaining misses are ambiguous, label-noisy cases.

Citation

@misc{prowl_secret_encoder,
  title        = {Prowl secret encoder},
  author       = {Prowl},
  year         = {2026},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/Podric/prowl-secret-encoder}
}

License

Noncommercial use only (CC BY-NC 4.0). Not for use in commercial products. Fine-tuned from distilbert-base-multilingual-cased (Apache-2.0). See the Prowl repository and the dataset card for data provenance and source licenses.