aparte-secrets

Finds passwords, API keys, tokens and private keys in text — in a 196 KB file, in the browser, with no API call.

aparte-secrets reads a chat message, a config file or source code and returns every secret it contains, with its exact position, its kind (password, api_key, token, private_key, other) and a confidence score. It is a token classifier of 168,597 parameters trained from scratch for this one task: no regex list, no entropy rule — the model reads the value and its context. It is meant to sit in front of an AI chat (mask a secret before the message leaves the device) and in a pre-commit hook.

Code, runtime, npm packages: github.com/apartejs/aparte-secrets-model · npm install @aparte/secrets · npx @aparte/secrets-cli
Benchmark: apartejs/aparte-secrets-gold · training data: apartejs/aparte-secrets-data

What changed in 1.1

About 4 times fewer false alerts on ordinary source code and docs: 0.69 → 0.17 per 1,000 lines on two public repositories with no secret. Version 1 flagged identifiers (maxTurns, createAgUiRunner), CSS values and words after an escaped \n in JSON files.
Better on chat: strict F1 0.69 → 0.76, passwords 0.74 → 0.82, false alerts in messages without a secret 17 → 11.
Same architecture and runtime. The packages now bundle int8 (about 196 KB): the int4 file of this version is noisier on code than its fp32 (see Files). The v1 files stay in v1/.

What it returns

Real outputs of secrets-v1.1-int8.bin on a fixed list of examples (span → subtype (score)), mistakes included:

input	found
`my db password is Hunter2024! can you check why the login fails?`	`Hunter2024!` → password (0.89)
`Le mot de passe du wifi de l'atelier c'est Biscotte19&, tu peux le noter ?`	`Biscotte19&` → password (1.00)
`the wifi password is Sunflower77# for the guests`	`Sunflower77#` → password (0.92)
`DB_HOST=localhost ⏎ DB_USER=app ⏎ DB_PASSWORD="t1?HQBFr=hHt_i+z9" ⏎ LOG_LEVEL=debug`	`t1?HQBFr=hHt_i+z9` → password (0.85)
`export OPENWEATHER_API_KEY=9b1c4e7f2a8d3b6c5e0f1a2b3c4d5e6f`	`9b1c4e7f2a8d3b6c5e0f1a2b3c4d5e6f` → api_key (0.99)
`J'ai oublié le mot de passe de l'imprimante, comment le réinitialiser ?`	`imprimante` → password (0.92)
`commit 3f2a9c1e8b7d6f5a4c3b2a1908f7e6d5c4b3a291, request id 123e4567-e89b-12d3-a456-426614174000`	nothing
`password: changeme # put your own here`	nothing

How it compares

Every system on the same test sets.

Chat — chat-fresh: 440 English and French chat messages written after training and never used to tune anything (396 secrets; 110 messages talk about a password without giving one). Strict span precision, recall and F1, recall on passwords and on API keys of providers never seen in training, false alerts in the messages without a secret:

system	precision	recall	F1	passwords	unknown providers	false alerts
aparte-secrets v1.1 (int8)	0.76	0.76	0.76	0.82	0.93	11 / 110
aparte-secrets v1 (int4)	0.70	0.69	0.69	0.74	0.79	17 / 110
OpenAI privacy-filter	0.52	0.54	0.53	0.64	0.95	55 / 110
detect-secrets 1.5.0	0.78	0.16	0.27	0.04	0.21	7 / 110
gitleaks 8.30.1	0.97	0.17	0.29	0.00	0.33	0 / 110

The same messages as JSON (escaped newlines and quotes, as in a log or an export, config chat-fresh-json):

system	precision	recall	F1	false alerts
aparte-secrets v1.1 (int8)	0.61	0.57	0.59	8 / 110
aparte-secrets v1 (int4)	0.55	0.53	0.54	22 / 110

Clean repositories — every tracked file of two public repositories that hold no secret (1,387 files, 218,493 lines; clean_repos.py rebuilds the set). Everything flagged is a false alert; the models at the CLI's default threshold (0.5):

system	false alerts	per 1,000 lines
aparte-secrets v1.1 (int8)	37	0.17
aparte-secrets v1 (int4)	150	0.69
detect-secrets 1.5.0	13	0.06
gitleaks 8.30.1	5	0.02

Long source files — 65 whole files (31,808 lines, 131 secrets) from the test repositories of CredData:

system	recall	false alerts per 1,000 lines	F1
aparte-secrets v1.1 (int8)	0.65	1.5	0.64
aparte-secrets v1 (int4)	0.56	1.7	0.57
detect-secrets 1.5.0	0.53	1.5	0.56
gitleaks 8.30.1	0.14	0.1	0.24

CredData — the lines of its test repositories (split by repository: no repository is seen in training), relabeled to our contract (public identifiers such as UUIDs are not secrets). Span F1, then line-level precision, recall and F1 (the metric of the CredData README):

system	span F1	line precision	line recall	line F1
aparte-secrets v1.1 (int8)	0.72	0.85	0.85	0.85
aparte-secrets v1 (int4)	0.74	0.88	0.84	0.86
OpenAI privacy-filter	0.42	0.63	0.88	0.73
detect-secrets 1.5.0	0.41	0.54	0.59	0.57
gitleaks 8.30.1	0.34	0.93	0.24	0.39

Development set — gold: 775 chat messages, snippets and framework configs. The training data was tuned while watching it, so it flatters us; reported for completeness:

system	strict F1	partial F1
aparte-secrets v1.1 (int8)	0.75	0.88
aparte-secrets v1 (int4)	0.68	0.81
OpenAI privacy-filter	0.47	0.75
detect-secrets 1.5.0	0.37	0.51
gitleaks 8.30.1	0.47	0.47

Files

file	size	chat F1	clean repos: false alerts / 1,000 lines	CredData span F1	dev F1
`secrets-v1.1-fp32.bin`	677 KB	0.76	0.17	0.72	0.75
`secrets-v1.1-int8.bin`	196 KB	0.76	0.17	0.72	0.75
`secrets-v1.1-int4.bin`	114 KB	0.74	0.44	0.71	0.72
`secrets-v1.1-int3.bin`	94 KB	0.73	0.22	0.70	0.67

int8 is the recommended precision and the one bundled in the packages: it matches fp32. int4 is smaller but noisier on ordinary code (0.44 false alerts per 1,000 lines against 0.17); a mixed file (int4 embeddings, int8 elsewhere) that keeps fp32 quality at about 160 KB is planned. Every file carries its identity in its JSON header (name, version, precision, architecture, labels); manifest.json lists every file of every version with its size and SHA-256; references/ holds the outputs every runtime must reproduce exactly. Older versions live in v1/, unchanged.

Use it

import { loadSecrets } from "@aparte/secrets";
const secrets = await loadSecrets();
secrets.detect("the wifi password is Sunflower77# for the guests");
const { text, secrets: map } = secrets.mask(message);   // send `text` to the AI, keep `map` on the device
secrets.unmask(answer, map);                              // put the values back in the answer

npx @aparte/secrets-cli              # scan the files git tracks
npx @aparte/secrets-cli --staged     # pre-commit: only the lines the next commit adds

The model reports raw spans; what to ignore in your project is your policy, applied after it (minScore, ignore, inline aparte-secrets:ignore, a baseline). In Python, the numpy reference reader runs any file: python reader.py secrets-v1.1-int8.bin "text".

Speed

The 450 chat messages of the benchmark (250 characters on average) one by one, then its code texts as one 3,550-line file; one CPU core, median of three rounds (bench.json):

runtime	device	ms per chat message	µs per character of code
JavaScript (node v24.14.0)	desktop, one core	12.9	86
Kotlin (OpenJDK 21)	desktop, one core	12.7	82
Swift 6.4 (release build)	desktop, one core	8.5	68
Kotlin (Android 15, ART)	mid-range Android phone (MediaTek Dimensity 7050)	55.5	436

The CLI spreads files over every core. Much slower than a regex scanner on a whole repository.

Limits

English and French only.
Still noisier than regex scanners on clean code (0.17 false alerts per 1,000 lines, see above), and less precise on escaped JSON than on plain text.
Boundaries can be imperfect (a trailing symbol, a punctuation-heavy value); the partial scores count a secret found with imperfect boundaries.
Unknown providers rely on context: a random-looking key is found when the text says what it is.
The chat sets and the gold are synthetic (secrets written into real WildChat messages by an LLM, configs generated by code); CredData and the clean repositories are the independent references.

License

Downloads last month: -

apartejs
/

aparte-secrets