README.md

---
license: cc-by-nc-4.0
base_model: distilbert-base-multilingual-cased
pipeline_tag: text-classification
library_name: transformers
language:
  - multilingual
  - en
  - de
  - fr
  - es
  - ru
tags:
  - security
  - secret-detection
  - credentials
  - dlp
  - code
metrics:
  - f1
  - precision
  - recall
---

<p align="center"><img src="https://huggingface.co/Podric/prowl-secret-encoder/resolve/main/logo.png" width="360" alt="Prowl"></p>

# Prowl secret encoder

A multilingual text classifier that scores whether a fragment of text (a line of code, a config
value, a Jira comment, a log line, a chat message) **contains a leaked credential**. It is stage 3
of [Prowl](https://github.com/Lercas/prowl), a high-precision secret scanner, and handles the
free-form and multilingual tail that regex and the linear model miss.

This model is **not a standalone scanner**. It is one stage of an ensemble; on its own it is a
recall booster, not a precision oracle. To scan a repo, use Prowl.

## What it does

- **Input:** a short text span (≤512 tokens; the tool feeds ~128).
- **Output:** two logits → `softmax(...)[1]` is P(text contains a secret).
- **Decision:** fires at `P ≥ 0.90`, a threshold calibrated on a held-out validation split to
  **precision ≥ 0.95** (value-disjoint from the benchmark, no leakage).

## Role in the ensemble

Prowl combines three stages by union: a Go regex/checksum/entropy cascade, a char+word TF-IDF
logistic regression, and this encoder. Adding the encoder to the other two, measured on
[ProwlBench](https://github.com/Lercas/prowlbench) (3,843 leakage-safe
cases):

| Configuration | Precision | Recall | F1 |
|---|:--:|:--:|:--:|
| cascade ∪ LR | 0.974 | 0.848 | 0.909 |
| **+ this encoder** | **0.971** | **0.970** | **0.970** |

The encoder lifts recall by 0.12 at a 0.003 precision cost, and reaches recall **1.00** on German,
French, Spanish, and Russian prose passwords - the free-form, multilingual tail that regex and the
linear model miss.

Standalone recall at a fixed precision of 0.97, by channel:

| code | Jira | Confluence | log |
|:--:|:--:|:--:|:--:|
| 0.984 | 0.996 | 0.998 | 0.947 |

## Usage

```python
import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tok = AutoTokenizer.from_pretrained("Podric/prowl-secret-encoder")
model = AutoModelForSequenceClassification.from_pretrained("Podric/prowl-secret-encoder").eval()

def secret_score(text: str) -> float:
    enc = tok(text, return_tensors="pt", truncation=True, max_length=128)
    with torch.no_grad():
        return torch.softmax(model(**enc).logits, -1)[0, 1].item()

secret_score('DB_PASSWORD = "tR4!nf0rce-2026-prod"')   # high
secret_score("the deployment finished without errors")  # low
# fire at >= 0.90
```

## Example output

`secret_score(text)` on a few inputs (the model fires at `P ≥ 0.90`):

| P(secret) | fires | input |
|:--:|:--:|---|
| **1.000** | yes | `DB_PASSWORD = "tR4!nf0rce-2026-prod"` |
| **1.000** | yes | `Ihr neues Passwort lautet F4QPE91sc6iN ...` (German prose) |
| **1.000** | yes | `Пароль от прод-сервера: Zx9!kLmN2k24qP ...` (Russian prose) |
| 0.001 | no | `token = os.environ["SERVICE_TOKEN"]` (env reference) |
| 0.002 | no | `Развёртывание завершилось без ошибок ...` (Russian log) |
| 0.000 | no | `def calculate_total(items): return sum(...)` (benign code) |
| 0.000 | no | `API_KEY=your_api_key_here` (placeholder) |

It fires on real passwords, including free-form prose in German and Russian that has no fixed prefix
for a regex to anchor, and stays near zero on benign code, logs, env-var references, and placeholders.

## Training

- **Base:** `distilbert-base-multilingual-cased` (104 languages, 6 layers, 134M params).
- **Objective:** binary sequence classification (secret-bearing / not).
- **Data:** the [Prowl secrets corpus](https://huggingface.co/datasets/Podric/prowl-secrets-corpus),
  503k labeled records across code, tickets, logs, and prose. Real-secret sources (CredData, HF
  PII sets) are held out by origin so the validation split is not a memorization check. Positives are
  oversampled and real-origin records up-weighted.
- **Calibration:** the operating threshold is fixed post-hoc on validation to a precision target, not
  learned, so it transfers to the ensemble's union rule.

## Limitations

- **Binary, not typed.** It says *secret / not secret*; the secret **type** (AWS vs Stripe vs ...)
  comes from Prowl's cascade.
- **A stage, not a scanner.** High recall comes at a precision that only makes sense inside the
  ensemble, where the cascade supplies structured-token precision. Do not deploy it alone as a gate.
- **Span-level, not token-level.** It flags a span; Prowl localizes the exact value.
- **Distilled size.** Chosen for CPU-friendly latency over a larger encoder; the few remaining misses
  are ambiguous, label-noisy cases.

## Citation

```bibtex
@misc{prowl_secret_encoder,
  title        = {Prowl secret encoder},
  author       = {Prowl},
  year         = {2026},
  howpublished = {Hugging Face},
  url          = {https://huggingface.co/Podric/prowl-secret-encoder}
}
```

## License

Noncommercial use only (CC BY-NC 4.0). Not for use in commercial products. Fine-tuned from
`distilbert-base-multilingual-cased` (Apache-2.0). See the
[Prowl repository](https://github.com/Lercas/prowl) and the dataset card for data provenance and
source licenses.