Mirror reviewed model release minilm-l6-v5-20260928

README.md

@@ -12,7 +12,38 @@ tags:
 
 [Botfilter](https://botfilter.io) is a browser extension that scores English text for “likely AI-written” as a calibrated probability, on the device. A score can be wrong and should not be used as proof of authorship.
 
-Each `releases/<version>/` directory contains the ONNX model, the WordPiece vocabulary, the calibration settings, and a lockfile with SHA-256 checksums. Use all files from the same version. The extension’s [inference implementation](https://github.com/hraness/botfilter/tree/main/extension/src) defines preprocessing, window aggregation, and calibration; these files do not provide a Transformers pipeline.
+Each `releases/<version>/` directory contains the ONNX model, the WordPiece vocabulary, the calibration settings, and a lockfile with SHA-256 checksums. Use all files from the same version. The extension’s [inference implementation](https://github.com/hraness/botfilter/tree/78d39e6debbff69b66661a760c0cca0f1b64a629/extension/src) defines preprocessing, window aggregation, and calibration; these files do not provide a Transformers pipeline.
+
+## minilm-l6-v5-20260928
+
+- Base: `nreimers/MiniLM-L6-H384-uncased` (MIT), fine-tuned as a two-class classifier. 22.7M parameters, weight-only int8, 23.6 MB.
+- Input: text folded with normalizer 2 (plain quotes and dashes, look-alike letters mapped to Latin), then WordPiece tokens in up to two 256-token windows, the start and the end of the text. The score is the mean logit margin across windows.
+- Output: `calibration.json` holds the temperature that turns the margin into a probability, and margin thresholds for three settings, Fewer, Balanced, and More, for texts of 25–80 words and longer texts. Texts under 25 words are not scored.
+
+### Training data
+
+Human text: the v4 sources (Common Pile news with per-document CC BY 4.0, Foodista CC BY 3.0, UK Hansard under the Open Parliament Licence, HH-RLHF MIT) plus owner-approved casual pools at pre-ChatGPT trust tiers: webis/tldr-17 Reddit 2006–2016 (CC BY 4.0), Enron email (FERC public record), Common Pile GitHub discussion (permissive repository licenses), and FineWeb pages (ODC-BY 1.0). AI text: 26,227 generations from eight Apache-2.0 or MIT open models run by the project (Qwen3, Phi-4, Mistral 7B and Small 24B, OLMo 2, SmolLM3) and seven API models (GPT-6 Sol, GPT-5.5, GPT-5.6 Terra, Claude Sonnet 5, Claude Opus 5.5, Claude Haiku 4.5, Mistral Medium 3.5), written in the same registers as the human text, including LinkedIn- and X-style posts.
+
+### Evaluation
+
+Sealed test split, Balanced setting, measured once after the thresholds were set:
+
+| Text | Flagged as likely AI-written |
+| --- | ---: |
+| Human news, blog, parliamentary, and chat text | 0.3–0.5% |
+| Human Reddit posts | 0.5% |
+| Human work email | 1.6% |
+| Human GitHub discussion and web pages | 1.2–1.8% |
+| AI text from the eight open models | 97% |
+| AI text from Granite 3.3 and Gemini 3.8 Flash, never trained on | 99% and 94% |
+| AI text from Claude Sonnet 5, Opus 5.5, Haiku 4.5 | 96% |
+| AI text from GPT-6 Sol, GPT-5.5, GPT-5.6 Terra | 96% |
+
+The [sealed evaluation results](https://github.com/hraness/botfilter/blob/78d39e6debbff69b66661a760c0cca0f1b64a629/model/results/minilm-l6-v5-holdout.json), [training recipe](https://github.com/hraness/botfilter/blob/78d39e6debbff69b66661a760c0cca0f1b64a629/model/results/minilm-l6-v5-recipe.json), and [source rights review](https://github.com/hraness/botfilter/blob/78d39e6debbff69b66661a760c0cca0f1b64a629/kb/notes/model-rights-v5-2026-09-28.md) describe this exact release. The source repository currently requires access; those evidence links are unavailable to anonymous readers.
+
+### Limits
+
+AI-written posts under 80 words are harder: 91–96% are flagged at Balanced. Human posts from X and LinkedIn were not available with suitable rights, so false-positive rates there are unmeasured. Text that a person and a model both edited is often missed. Non-English text is out of scope.
 
 ## minilm-l6-v4-20260928
 
@@ -47,4 +78,6 @@ AI-written posts under 80 words are harder: 82% of short AI social posts are fla
 
 Contains Parliamentary information licensed under the [Open Parliament Licence v3.0](https://www.parliament.uk/site-information/copyright-parliament/open-parliament-licence/). Trained on CC BY 4.0 news articles from the [Common Pile news collection](https://huggingface.co/datasets/common-pile/news) and CC BY 3.0 posts by [Foodista](https://huggingface.co/datasets/common-pile/foodista) contributors, and on [Anthropic HH-RLHF](https://huggingface.co/datasets/Anthropic/hh-rlhf) (MIT).
 
+The v5 model also uses the Webis group’s [TL;DR corpus](https://huggingface.co/datasets/webis/tldr-17) (CC BY 4.0), the [Enron email corpus](https://www.cs.cmu.edu/~enron/) (FERC public record), [Common Pile GitHub discussions](https://huggingface.co/datasets/common-pile/github_archive) from repositories with permissive licenses, and [FineWeb](https://huggingface.co/datasets/HuggingFaceFW/fineweb) by Hugging Face (ODC-BY 1.0). These casual sources were admitted under their distributors’ licenses or releases with the owner’s approval; upstream author-level terms were not individually cleared. Only trained model files are distributed here.
+
 [Source and extension](https://github.com/hraness/botfilter) · [Project website](https://botfilter.io)

releases/minilm-l6-v5-20260928/botfilter.int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2ea44653426db0fdc73981f17cae8f22493072b8df31f6ed2c291928065e1812
+size 23626443

releases/minilm-l6-v5-20260928/calibration.json

@@ -0,0 +1,20 @@
+{
+ "modelVersion": "minilm-l6-v5-20260928",
+ "temperature": 1.3289,
+ "minWords": 25,
+ "maxTokens": 256,
+ "shortMaxWords": 80,
+ "thresholds": {
+  "short": {
+   "fewer": 4.2972,
+   "balanced": 3.1868,
+   "more": 0.346
+  },
+  "long": {
+   "fewer": 2.8064,
+   "balanced": 1.1287,
+   "more": -1.7843
+  }
+ },
+ "normalizer": 2
+}

releases/minilm-l6-v5-20260928/model.lock.json

@@ -0,0 +1,22 @@
+{
+  "version": "minilm-l6-v5-20260928",
+  "release": "model-minilm-l6-v5-20260928",
+  "files": {
+    "vocab.txt": {
+      "sha256": "07eced375cec144d27c900241f3e339478dec958f92fddbc551f295c992038a3",
+      "bytes": 231508
+    },
+    "calibration.json": {
+      "sha256": "be2780a467e91657e4584e6d2345486bec6fc7910f0cdda7b489db5b0c314da4",
+      "bytes": 321
+    },
+    "botfilter.int8.onnx": {
+      "sha256": "2ea44653426db0fdc73981f17cae8f22493072b8df31f6ed2c291928065e1812",
+      "bytes": 23626443
+    },
+    "fixtures.json": {
+      "sha256": "2f6f7e1dd95e1558747e84aa63c8c81a63be52b11253de7f22c3d8183bb4ccaf",
+      "bytes": 65647
+    }
+  }
+}