Spaces:
Running
Running
bluepencil: a calibrated quality gate for prose, running entirely in the browser
Browse files- README.md +49 -5
- bluepencil.js +210 -0
- gates.json +233 -0
- index.html +216 -17
README.md
CHANGED
|
@@ -1,10 +1,54 @@
|
|
| 1 |
---
|
| 2 |
-
title:
|
| 3 |
-
emoji:
|
| 4 |
-
colorFrom:
|
| 5 |
-
colorTo:
|
| 6 |
sdk: static
|
|
|
|
| 7 |
pinned: false
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
---
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
title: bluepencil
|
| 3 |
+
emoji: ✏️
|
| 4 |
+
colorFrom: indigo
|
| 5 |
+
colorTo: gray
|
| 6 |
sdk: static
|
| 7 |
+
app_file: index.html
|
| 8 |
pinned: false
|
| 9 |
+
license: mit
|
| 10 |
+
short_description: A prose quality gate with a measured 5% false-positive rate
|
| 11 |
+
tags:
|
| 12 |
+
- text-classification
|
| 13 |
+
- writing
|
| 14 |
+
- ai-detection
|
| 15 |
+
- evaluation
|
| 16 |
+
- linter
|
| 17 |
+
- prose
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# bluepencil
|
| 21 |
+
|
| 22 |
+
A quality gate for prose. Paste a draft, see which gates fire and why.
|
| 23 |
+
|
| 24 |
+
It fails 46.9% of machine-written documents and 6.3% of human-written ones. Both numbers were measured on 1,500 held-out documents per class rather than asserted.
|
| 25 |
+
|
| 26 |
+
## What makes it different from the other detectors on the Hub
|
| 27 |
+
|
| 28 |
+
Most tools of this kind give you a percentage and no way to check it. This one reports named findings with line numbers and a fix for each, and every threshold behind those findings was calibrated rather than chosen.
|
| 29 |
+
|
| 30 |
+
The method: measure each statistic across 2,602 human-written documents spanning three registers, informal forum answers, formal scientific abstracts, and multi-paragraph encyclopedia articles, then place the threshold at the percentile that produces a 5% false-positive rate. The panel in the app shows you the measured value against the threshold for every cadence gate, so you can disagree with it on the evidence.
|
| 31 |
+
|
| 32 |
+
Eighteen gates in two families. Pattern gates point at a span of text. Cadence gates measure the whole document, and those are the ones that catch prose which had every flagged word swapped out and still reads flat.
|
| 33 |
+
|
| 34 |
+
Everything runs in your browser. Nothing you paste is uploaded anywhere.
|
| 35 |
+
|
| 36 |
+
## What it will not tell you
|
| 37 |
+
|
| 38 |
+
Whether the text is true, whether the argument holds, or who wrote it. It measures the surface. A document can pass every gate and still be empty.
|
| 39 |
+
|
| 40 |
+
It is a gate, not a classifier. Failing means the text contains constructions a particular editorial standard rejects, which is true of plenty of good human writing, as the 6.3% says.
|
| 41 |
+
|
| 42 |
+
Nine of the eighteen gates fire more often on human writing than machine writing on the validation corpus. That is published rather than hidden, along with the reason: the validation corpus was collected around December 2022 and cannot contain constructions that only became common later.
|
| 43 |
+
|
| 44 |
+
## Elsewhere
|
| 45 |
+
|
| 46 |
+
The CLI, the pre-commit hook, the GitHub Action, and the agent skill: [github.com/wolfvswhale/bluepencil](https://github.com/wolfvswhale/bluepencil)
|
| 47 |
+
|
| 48 |
+
The evaluation harness that produced the thresholds, including its own negative results: [github.com/wolfvswhale/prose-eval](https://github.com/wolfvswhale/prose-eval)
|
| 49 |
+
|
| 50 |
+
The measurements underneath it all: [prose-cadence-stats](https://huggingface.co/datasets/wolfvswhale/prose-cadence-stats)
|
| 51 |
+
|
| 52 |
+
Rules derived from Wikipedia's [Signs of AI writing](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing), maintained by WikiProject AI Cleanup, plus a house editing standard.
|
| 53 |
+
|
| 54 |
+
Built by J. Alderman Lyell. MIT licensed.
|
bluepencil.js
ADDED
|
@@ -0,0 +1,210 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
/* bluepencil, ported to run in the browser.
|
| 2 |
+
*
|
| 3 |
+
* The Python package is the reference implementation. This is a faithful port
|
| 4 |
+
* of the scoring logic so the Space can be static, which on Hugging Face is
|
| 5 |
+
* the only tier that is free. It also means nothing you paste leaves your
|
| 6 |
+
* machine, and there is no cold start.
|
| 7 |
+
*
|
| 8 |
+
* Gate definitions and every calibrated threshold are loaded from gates.json,
|
| 9 |
+
* which is generated from the Python package rather than retyped, so the two
|
| 10 |
+
* cannot drift.
|
| 11 |
+
*/
|
| 12 |
+
|
| 13 |
+
const TRAILING_WS = /[ \t]+$/;
|
| 14 |
+
|
| 15 |
+
/* ---------- markdown stripping -------------------------------------------
|
| 16 |
+
* Replace non-prose spans with spaces, preserving newlines and total length,
|
| 17 |
+
* so character offsets still map to the right line and column.
|
| 18 |
+
*/
|
| 19 |
+
function blank(match) {
|
| 20 |
+
return match.replace(/[^\n]/g, " ");
|
| 21 |
+
}
|
| 22 |
+
|
| 23 |
+
function stripNonProse(text) {
|
| 24 |
+
let out = text;
|
| 25 |
+
const sub = (re) => { out = out.replace(re, blank); };
|
| 26 |
+
|
| 27 |
+
sub(/^---\r?\n[\s\S]*?\r?\n---[ \t]*\r?\n/); // YAML front matter
|
| 28 |
+
// fenced code: pair up fences
|
| 29 |
+
const fence = /^([ \t]*)(`{3,}|~{3,}).*$/gm;
|
| 30 |
+
const fences = [...out.matchAll(fence)];
|
| 31 |
+
for (let i = 0; i + 1 < fences.length; i += 2) {
|
| 32 |
+
const start = fences[i].index;
|
| 33 |
+
const end = fences[i + 1].index + fences[i + 1][0].length;
|
| 34 |
+
out = out.slice(0, start) + blank(out.slice(start, end)) + out.slice(end);
|
| 35 |
+
}
|
| 36 |
+
const odd = [...out.matchAll(fence)];
|
| 37 |
+
if (odd.length % 2 === 1) {
|
| 38 |
+
const s = odd[odd.length - 1].index;
|
| 39 |
+
out = out.slice(0, s) + blank(out.slice(s));
|
| 40 |
+
}
|
| 41 |
+
|
| 42 |
+
sub(/!\[[^\]]*\]\([^)]*\)/g); // images
|
| 43 |
+
sub(/\]\([^)\s]+(?:\s+"[^"]*")?\)/g); // link targets
|
| 44 |
+
sub(/`[^`\n]+`/g); // inline code
|
| 45 |
+
sub(/https?:\/\/\S+/g); // bare urls
|
| 46 |
+
sub(/<[^>\n]{1,120}>/g); // html tags
|
| 47 |
+
sub(/^[ \t]{0,3}\|.*\|[ \t]*$/gm); // table rows
|
| 48 |
+
sub(/^[ \t]{0,3}#{1,6}[ \t]+.*$/gm); // headings
|
| 49 |
+
sub(/^[ \t]{0,3}>.*$/gm); // blockquotes
|
| 50 |
+
return out;
|
| 51 |
+
}
|
| 52 |
+
|
| 53 |
+
/* ---------- parsing ------------------------------------------------------ */
|
| 54 |
+
|
| 55 |
+
const WORD_RE = /[A-Za-z0-9][A-Za-z0-9'’-]*/g;
|
| 56 |
+
const SENT_SPLIT = /(?<=[.!?])["')\]]*\s+/;
|
| 57 |
+
|
| 58 |
+
function words(s) { return s.match(WORD_RE) || []; }
|
| 59 |
+
|
| 60 |
+
function parse(raw) {
|
| 61 |
+
const text = stripNonProse(raw);
|
| 62 |
+
const lineStarts = [0];
|
| 63 |
+
for (let i = 0; i < raw.length; i++) if (raw[i] === "\n") lineStarts.push(i + 1);
|
| 64 |
+
|
| 65 |
+
const paragraphs = [];
|
| 66 |
+
let offset = 0;
|
| 67 |
+
for (const chunk of text.split(/\n[ \t]*\n/)) {
|
| 68 |
+
const trimmed = chunk.trim();
|
| 69 |
+
if (trimmed) {
|
| 70 |
+
const sentences = [];
|
| 71 |
+
let cursor = 0;
|
| 72 |
+
for (const piece of chunk.split(SENT_SPLIT)) {
|
| 73 |
+
const t = piece.trim();
|
| 74 |
+
if (!t) continue;
|
| 75 |
+
let idx = chunk.indexOf(t, cursor);
|
| 76 |
+
if (idx === -1) idx = cursor;
|
| 77 |
+
const w = words(t);
|
| 78 |
+
if (w.length) sentences.push({ text: t, start: offset + idx, words: w });
|
| 79 |
+
cursor = idx + t.length;
|
| 80 |
+
}
|
| 81 |
+
paragraphs.push({ text: trimmed, sentences });
|
| 82 |
+
}
|
| 83 |
+
offset += chunk.length + 2;
|
| 84 |
+
}
|
| 85 |
+
|
| 86 |
+
const sentences = paragraphs.flatMap((p) => p.sentences);
|
| 87 |
+
return { raw, text, lineStarts, paragraphs, sentences, words: words(text) };
|
| 88 |
+
}
|
| 89 |
+
|
| 90 |
+
function position(doc, offset) {
|
| 91 |
+
let lo = 0, hi = doc.lineStarts.length - 1;
|
| 92 |
+
while (lo < hi) {
|
| 93 |
+
const mid = Math.ceil((lo + hi) / 2);
|
| 94 |
+
if (doc.lineStarts[mid] <= offset) lo = mid; else hi = mid - 1;
|
| 95 |
+
}
|
| 96 |
+
return { line: lo + 1, column: offset - doc.lineStarts[lo] + 1 };
|
| 97 |
+
}
|
| 98 |
+
|
| 99 |
+
/* ---------- statistics --------------------------------------------------- */
|
| 100 |
+
|
| 101 |
+
const mean = (a) => a.reduce((x, y) => x + y, 0) / a.length;
|
| 102 |
+
function pstdev(a) {
|
| 103 |
+
const m = mean(a);
|
| 104 |
+
return Math.sqrt(a.reduce((s, v) => s + (v - m) ** 2, 0) / a.length);
|
| 105 |
+
}
|
| 106 |
+
function cv(a) {
|
| 107 |
+
if (a.length < 2) return 0;
|
| 108 |
+
const m = mean(a);
|
| 109 |
+
return m === 0 ? 0 : pstdev(a) / m;
|
| 110 |
+
}
|
| 111 |
+
|
| 112 |
+
const MEASURES = {
|
| 113 |
+
"flat-cadence": (d) => {
|
| 114 |
+
const l = d.sentences.map((s) => s.words.length);
|
| 115 |
+
return l.length < 4 ? null : cv(l);
|
| 116 |
+
},
|
| 117 |
+
"uniform-paragraphs": (d) => {
|
| 118 |
+
const c = d.paragraphs.filter((p) => p.sentences.length).map((p) => p.sentences.length);
|
| 119 |
+
return c.length < 3 ? null : cv(c);
|
| 120 |
+
},
|
| 121 |
+
"no-short-sentences": (d) => {
|
| 122 |
+
const l = d.sentences.map((s) => s.words.length);
|
| 123 |
+
return l.length < 4 ? null : l.filter((n) => n <= 8).length / l.length;
|
| 124 |
+
},
|
| 125 |
+
"triplet-habit": (d) => {
|
| 126 |
+
if (d.words.length < 60) return null;
|
| 127 |
+
const m = d.text.match(/\b[\w'-]+,\s+[\w'-]+,\s+(?:and|or)\s+[\w'-]+/g) || [];
|
| 128 |
+
return (m.length * 1000) / d.words.length;
|
| 129 |
+
},
|
| 130 |
+
"em-dash-habit": (d) => {
|
| 131 |
+
if (d.words.length < 60) return null;
|
| 132 |
+
return ((d.text.split("—").length - 1) * 1000) / d.words.length;
|
| 133 |
+
},
|
| 134 |
+
"repeated-openers": (d) => {
|
| 135 |
+
const starts = d.sentences.filter((s) => s.words.length).map((s) => s.words[0].toLowerCase());
|
| 136 |
+
if (starts.length < 5) return null;
|
| 137 |
+
let r = 0;
|
| 138 |
+
for (let i = 1; i < starts.length; i++) if (starts[i] === starts[i - 1]) r++;
|
| 139 |
+
return r / (starts.length - 1);
|
| 140 |
+
},
|
| 141 |
+
};
|
| 142 |
+
|
| 143 |
+
/* ---------- the check ---------------------------------------------------- */
|
| 144 |
+
|
| 145 |
+
export function check(raw, spec, minWords = 120) {
|
| 146 |
+
const doc = parse(raw);
|
| 147 |
+
const findings = [];
|
| 148 |
+
|
| 149 |
+
for (const gate of spec.patterns) {
|
| 150 |
+
if (gate.severity === "off") continue;
|
| 151 |
+
let hits = [];
|
| 152 |
+
for (const src of gate.patterns) {
|
| 153 |
+
const re = new RegExp(src, "gi");
|
| 154 |
+
for (const m of doc.text.matchAll(re)) hits.push({ start: m.index, span: m[0] });
|
| 155 |
+
}
|
| 156 |
+
if (!hits.length) continue;
|
| 157 |
+
|
| 158 |
+
// de-duplicate overlapping matches from sibling patterns
|
| 159 |
+
hits.sort((a, b) => a.start - b.start);
|
| 160 |
+
const kept = [];
|
| 161 |
+
let lastEnd = -1;
|
| 162 |
+
for (const h of hits) {
|
| 163 |
+
if (h.start >= lastEnd) { kept.push(h); lastEnd = h.start + h.span.length; }
|
| 164 |
+
}
|
| 165 |
+
|
| 166 |
+
if (gate.budget > 0 && doc.words.length) {
|
| 167 |
+
const allowed = Math.floor((gate.budget * doc.words.length) / 1000);
|
| 168 |
+
if (kept.length <= allowed) continue;
|
| 169 |
+
}
|
| 170 |
+
|
| 171 |
+
for (const h of kept) {
|
| 172 |
+
const { line, column } = position(doc, h.start);
|
| 173 |
+
findings.push({
|
| 174 |
+
gate: gate.name, severity: gate.severity, kind: "pattern",
|
| 175 |
+
message: gate.description, suggestion: gate.suggestion,
|
| 176 |
+
excerpt: h.span.replace(/\s+/g, " ").slice(0, 90),
|
| 177 |
+
line, column,
|
| 178 |
+
});
|
| 179 |
+
}
|
| 180 |
+
}
|
| 181 |
+
|
| 182 |
+
const cadence = [];
|
| 183 |
+
for (const gate of spec.cadence) {
|
| 184 |
+
const measure = MEASURES[gate.name];
|
| 185 |
+
const value = doc.words.length >= Math.max(gate.min_words, minWords) && measure ? measure(doc) : null;
|
| 186 |
+
const fires = gate.threshold !== null && value !== null &&
|
| 187 |
+
(gate.direction === "low" ? value < gate.threshold : value > gate.threshold);
|
| 188 |
+
cadence.push({ ...gate, value, fires, measurable: value !== null });
|
| 189 |
+
if (fires && gate.severity !== "off") {
|
| 190 |
+
const cmp = gate.direction === "low" ? "below" : "above";
|
| 191 |
+
findings.push({
|
| 192 |
+
gate: gate.name, severity: gate.severity, kind: "cadence",
|
| 193 |
+
message: `${gate.description} Measured ${value.toFixed(3)}, ${cmp} the calibrated threshold of ${gate.threshold.toFixed(3)}.`,
|
| 194 |
+
suggestion: gate.suggestion, excerpt: "", line: 0, column: 0,
|
| 195 |
+
});
|
| 196 |
+
}
|
| 197 |
+
}
|
| 198 |
+
|
| 199 |
+
const rank = { error: 2, warn: 1, off: 0 };
|
| 200 |
+
findings.sort((a, b) => (rank[b.severity] - rank[a.severity]) || (a.line - b.line) || (a.column - b.column));
|
| 201 |
+
|
| 202 |
+
return {
|
| 203 |
+
findings, cadence,
|
| 204 |
+
words: doc.words.length,
|
| 205 |
+
sentences: doc.sentences.length,
|
| 206 |
+
paragraphs: doc.paragraphs.length,
|
| 207 |
+
errors: findings.filter((f) => f.severity === "error").length,
|
| 208 |
+
warnings: findings.filter((f) => f.severity === "warn").length,
|
| 209 |
+
};
|
| 210 |
+
}
|
gates.json
ADDED
|
@@ -0,0 +1,233 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"patterns": [
|
| 3 |
+
{
|
| 4 |
+
"name": "negative-parallelism",
|
| 5 |
+
"description": "'Not just X, it's Y' and its relatives. The most recognizable machine construction.",
|
| 6 |
+
"suggestion": "State the correct thing. Leave the wrong thing out.",
|
| 7 |
+
"severity": "error",
|
| 8 |
+
"budget": 0.0,
|
| 9 |
+
"patterns": [
|
| 10 |
+
"not just [^.!?]{1,50}?,? (?:but |it'?s |it is )",
|
| 11 |
+
"not only\\b[^.!?]{0,60}?\\bbut also\\b",
|
| 12 |
+
"(?:isn'?t|is not) (?:about|just)\\b[^.!?]{0,50}?[;,] it'?s",
|
| 13 |
+
"it'?s not\\b[^.!?]{0,40}?[;,] it'?s\\b"
|
| 14 |
+
]
|
| 15 |
+
},
|
| 16 |
+
{
|
| 17 |
+
"name": "inflated-significance",
|
| 18 |
+
"description": "Announcing that something matters instead of showing what it did.",
|
| 19 |
+
"suggestion": "Show what it did, with a fact. Cut the claim of importance.",
|
| 20 |
+
"severity": "error",
|
| 21 |
+
"budget": 0.0,
|
| 22 |
+
"patterns": [
|
| 23 |
+
"stands? as a testament",
|
| 24 |
+
"(?:is|are) a testament to",
|
| 25 |
+
"plays? a (?:vital|pivotal|crucial|key|significant) role",
|
| 26 |
+
"underscor\\w+ (?:its|the|their) (?:importance|significance)",
|
| 27 |
+
"leav\\w+ a lasting (?:impact|impression|legacy)",
|
| 28 |
+
"watershed moment",
|
| 29 |
+
"cannot be overstated",
|
| 30 |
+
"solidif\\w+ (?:its|their) (?:position|place|status)"
|
| 31 |
+
]
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"name": "participle-tail",
|
| 35 |
+
"description": "A clause tacked onto a sentence to fake analysis.",
|
| 36 |
+
"suggestion": "Explain the consequence concretely, or stop at the fact.",
|
| 37 |
+
"severity": "error",
|
| 38 |
+
"budget": 0.0,
|
| 39 |
+
"patterns": [
|
| 40 |
+
",\\s+(?:highlighting|underscoring|illustrating|reflecting|emphasizing|showcasing|paving|cementing|signaling|marking)\\b[^.!?]{0,80}[.!?]"
|
| 41 |
+
]
|
| 42 |
+
},
|
| 43 |
+
{
|
| 44 |
+
"name": "editorializing",
|
| 45 |
+
"description": "Telling the reader that something is worth noting instead of noting it.",
|
| 46 |
+
"suggestion": "Just write the note.",
|
| 47 |
+
"severity": "error",
|
| 48 |
+
"budget": 0.0,
|
| 49 |
+
"patterns": [
|
| 50 |
+
"it'?s important to (?:note|remember|understand|consider)",
|
| 51 |
+
"it is important to (?:note|remember|understand|consider)",
|
| 52 |
+
"it'?s worth (?:noting|remembering|mentioning)",
|
| 53 |
+
"no discussion would be complete without",
|
| 54 |
+
"needless to say"
|
| 55 |
+
]
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"name": "hollow-conclusion",
|
| 59 |
+
"description": "Restating a passage the reader just finished.",
|
| 60 |
+
"suggestion": "End on the last real thing there is to say. A piece can just stop.",
|
| 61 |
+
"severity": "error",
|
| 62 |
+
"budget": 0.0,
|
| 63 |
+
"patterns": [
|
| 64 |
+
"\\bin (?:summary|conclusion)\\b",
|
| 65 |
+
"(?:^|\\n)\\s*overall,",
|
| 66 |
+
"\\bat the end of the day\\b",
|
| 67 |
+
"\\bto sum up\\b"
|
| 68 |
+
]
|
| 69 |
+
},
|
| 70 |
+
{
|
| 71 |
+
"name": "throat-clearing",
|
| 72 |
+
"description": "Warming up instead of starting.",
|
| 73 |
+
"suggestion": "Delete it. Start with the first real sentence.",
|
| 74 |
+
"severity": "error",
|
| 75 |
+
"budget": 0.0,
|
| 76 |
+
"patterns": [
|
| 77 |
+
"in today'?s (?:fast-paced|modern|digital|ever-changing) world",
|
| 78 |
+
"in the (?:ever-)?(?:evolving|changing) (?:landscape|world) of",
|
| 79 |
+
"let'?s (?:dive in|unpack|explore this)",
|
| 80 |
+
"here'?s a comprehensive"
|
| 81 |
+
]
|
| 82 |
+
},
|
| 83 |
+
{
|
| 84 |
+
"name": "engagement-bait",
|
| 85 |
+
"description": "Nudging the reader instead of informing them.",
|
| 86 |
+
"suggestion": "Cut it.",
|
| 87 |
+
"severity": "error",
|
| 88 |
+
"budget": 0.0,
|
| 89 |
+
"patterns": [
|
| 90 |
+
"let that sink in",
|
| 91 |
+
"read that again",
|
| 92 |
+
"this changes everything",
|
| 93 |
+
"here'?s the thing\\b"
|
| 94 |
+
]
|
| 95 |
+
},
|
| 96 |
+
{
|
| 97 |
+
"name": "vague-attribution",
|
| 98 |
+
"description": "Claims with no named source.",
|
| 99 |
+
"suggestion": "Name the person, paper, or outlet with a date, or drop the claim.",
|
| 100 |
+
"severity": "error",
|
| 101 |
+
"budget": 0.0,
|
| 102 |
+
"patterns": [
|
| 103 |
+
"\\bstudies (?:show|suggest|indicate)\\b",
|
| 104 |
+
"\\bexperts (?:say|agree|note)\\b",
|
| 105 |
+
"\\bresearch (?:shows|suggests|indicates)\\b",
|
| 106 |
+
"\\b(?:some|many) (?:critics|observers|analysts) (?:argue|note|suggest)\\b",
|
| 107 |
+
"\\bindustry reports suggest\\b"
|
| 108 |
+
]
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"name": "promotional",
|
| 112 |
+
"description": "Brochure register.",
|
| 113 |
+
"suggestion": "Replace the adjective with a fact, or cut it.",
|
| 114 |
+
"severity": "warn",
|
| 115 |
+
"budget": 0.0,
|
| 116 |
+
"patterns": [
|
| 117 |
+
"rich (?:cultural )?(?:heritage|history|tapestry)",
|
| 118 |
+
"stunning natural beauty",
|
| 119 |
+
"nestled in the heart of",
|
| 120 |
+
"must-(?:visit|see|read|have)\\b",
|
| 121 |
+
"\\bbreathtaking\\b",
|
| 122 |
+
"\\bworld-class\\b"
|
| 123 |
+
]
|
| 124 |
+
},
|
| 125 |
+
{
|
| 126 |
+
"name": "transition-crutch",
|
| 127 |
+
"description": "Connective tissue that can usually be deleted with no loss.",
|
| 128 |
+
"suggestion": "Delete it, or use a plain connector: but, so, then.",
|
| 129 |
+
"severity": "warn",
|
| 130 |
+
"budget": 4.081,
|
| 131 |
+
"patterns": [
|
| 132 |
+
"\\bmoreover\\b",
|
| 133 |
+
"\\bfurthermore\\b",
|
| 134 |
+
"\\badditionally\\b",
|
| 135 |
+
"\\bon top of that\\b",
|
| 136 |
+
"\\bthat said\\b"
|
| 137 |
+
]
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"name": "copula-avoidance",
|
| 141 |
+
"description": "Swapping 'is' for something weightier.",
|
| 142 |
+
"suggestion": "Use 'is' or 'was'.",
|
| 143 |
+
"severity": "warn",
|
| 144 |
+
"budget": 0.0,
|
| 145 |
+
"patterns": [
|
| 146 |
+
"\\bserves? as\\b",
|
| 147 |
+
"\\bstands? as\\b",
|
| 148 |
+
"\\brepresents? a\\b"
|
| 149 |
+
]
|
| 150 |
+
},
|
| 151 |
+
{
|
| 152 |
+
"name": "assistant-residue",
|
| 153 |
+
"description": "Chat-assistant phrasing left in published text.",
|
| 154 |
+
"suggestion": "Delete. This should never reach a reader.",
|
| 155 |
+
"severity": "error",
|
| 156 |
+
"budget": 0.0,
|
| 157 |
+
"patterns": [
|
| 158 |
+
"\\bas an AI language model\\b",
|
| 159 |
+
"\\bI hope this helps\\b",
|
| 160 |
+
"\\blet me know if you (?:need|have)\\b",
|
| 161 |
+
"\\bfeel free to reach out\\b",
|
| 162 |
+
"\\bas of my (?:last )?knowledge (?:update|cutoff)\\b",
|
| 163 |
+
"\\bI hope this (?:message|email) finds you well\\b"
|
| 164 |
+
]
|
| 165 |
+
}
|
| 166 |
+
],
|
| 167 |
+
"cadence": [
|
| 168 |
+
{
|
| 169 |
+
"name": "flat-cadence",
|
| 170 |
+
"description": "Sentence lengths vary less than human writing typically does.",
|
| 171 |
+
"suggestion": "Rewrite one sentence in every three-sentence run to a different length. A short sentence after two long ones does more than any word swap.",
|
| 172 |
+
"severity": "warn",
|
| 173 |
+
"direction": "low",
|
| 174 |
+
"threshold": 0.2367,
|
| 175 |
+
"min_words": 120
|
| 176 |
+
},
|
| 177 |
+
{
|
| 178 |
+
"name": "uniform-paragraphs",
|
| 179 |
+
"description": "Paragraphs are all about the same length.",
|
| 180 |
+
"suggestion": "Break one paragraph early and let another run long. A one-line paragraph is allowed.",
|
| 181 |
+
"severity": "warn",
|
| 182 |
+
"direction": "low",
|
| 183 |
+
"threshold": 0.5227,
|
| 184 |
+
"min_words": 120
|
| 185 |
+
},
|
| 186 |
+
{
|
| 187 |
+
"name": "no-short-sentences",
|
| 188 |
+
"description": "Nothing in the draft is short.",
|
| 189 |
+
"suggestion": "Cut one sentence down to under eight words. It changes the rhythm of everything around it.",
|
| 190 |
+
"severity": "warn",
|
| 191 |
+
"direction": "low",
|
| 192 |
+
"threshold": null,
|
| 193 |
+
"min_words": 120
|
| 194 |
+
},
|
| 195 |
+
{
|
| 196 |
+
"name": "triplet-habit",
|
| 197 |
+
"description": "Reflexive lists of three: 'x, y, and z' over and over.",
|
| 198 |
+
"suggestion": "Use the number of items that exists. Two is fine. One is often best.",
|
| 199 |
+
"severity": "warn",
|
| 200 |
+
"direction": "high",
|
| 201 |
+
"threshold": 5.1787,
|
| 202 |
+
"min_words": 120
|
| 203 |
+
},
|
| 204 |
+
{
|
| 205 |
+
"name": "em-dash-habit",
|
| 206 |
+
"description": "Em dashes used as a default connector.",
|
| 207 |
+
"suggestion": "Use a comma, a colon, parentheses, or two sentences.",
|
| 208 |
+
"severity": "warn",
|
| 209 |
+
"direction": "high",
|
| 210 |
+
"threshold": 1.4888,
|
| 211 |
+
"min_words": 120
|
| 212 |
+
},
|
| 213 |
+
{
|
| 214 |
+
"name": "repeated-openers",
|
| 215 |
+
"description": "Consecutive sentences starting with the same word.",
|
| 216 |
+
"suggestion": "Vary the opener, unless the repetition is deliberate.",
|
| 217 |
+
"severity": "warn",
|
| 218 |
+
"direction": "high",
|
| 219 |
+
"threshold": 0.2222,
|
| 220 |
+
"min_words": 120
|
| 221 |
+
}
|
| 222 |
+
],
|
| 223 |
+
"meta": {
|
| 224 |
+
"calibrated": true,
|
| 225 |
+
"corpus": "HC3 human answers (informal) + human scientific abstracts (formal) + wikitext-103 articles (multi-paragraph)",
|
| 226 |
+
"n_documents": 2602,
|
| 227 |
+
"informal": 1000,
|
| 228 |
+
"formal": 1000,
|
| 229 |
+
"structured": 602,
|
| 230 |
+
"min_words": 120,
|
| 231 |
+
"target_fpr": 0.05
|
| 232 |
+
}
|
| 233 |
+
}
|
index.html
CHANGED
|
@@ -1,19 +1,218 @@
|
|
| 1 |
<!doctype html>
|
| 2 |
-
<html>
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 19 |
</html>
|
|
|
|
| 1 |
<!doctype html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="utf-8">
|
| 5 |
+
<meta name="viewport" content="width=device-width, initial-scale=1">
|
| 6 |
+
<title>bluepencil — a calibrated quality gate for prose</title>
|
| 7 |
+
<style>
|
| 8 |
+
:root {
|
| 9 |
+
--surface: #fcfcfb; --panel: #ffffff; --line: #e6e6e0;
|
| 10 |
+
--ink: #0b0b0b; --ink2: #52514e; --muted: #8a8a85;
|
| 11 |
+
--error: #b4232a; --warn: #a06010; --ok: #2e7d32; --accent: #2a78d6;
|
| 12 |
+
}
|
| 13 |
+
@media (prefers-color-scheme: dark) {
|
| 14 |
+
:root {
|
| 15 |
+
--surface: #16161a; --panel: #1d1d21; --line: #2e2e34;
|
| 16 |
+
--ink: #f2f2ef; --ink2: #b8b7b1; --muted: #7e7e86;
|
| 17 |
+
--error: #ff6b6b; --warn: #e0a34a; --ok: #6cc46c; --accent: #6aa9f0;
|
| 18 |
+
}
|
| 19 |
+
}
|
| 20 |
+
* { box-sizing: border-box; }
|
| 21 |
+
body {
|
| 22 |
+
margin: 0; background: var(--surface); color: var(--ink);
|
| 23 |
+
font: 15px/1.6 ui-sans-serif, -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
|
| 24 |
+
}
|
| 25 |
+
.wrap { max-width: 1180px; margin: 0 auto; padding: 26px 20px 60px; }
|
| 26 |
+
header { border-bottom: 2px solid var(--ink); padding-bottom: 14px; margin-bottom: 20px; }
|
| 27 |
+
h1 { font-size: 25px; margin: 0 0 6px; letter-spacing: -0.02em; }
|
| 28 |
+
.lede { color: var(--ink2); font-size: 15px; margin: 0; max-width: 74ch; }
|
| 29 |
+
.lede b { color: var(--ink); }
|
| 30 |
+
.meta { color: var(--muted); font-size: 13px; margin-top: 9px; }
|
| 31 |
+
.meta a { color: var(--accent); text-decoration: none; }
|
| 32 |
+
.meta a:hover { text-decoration: underline; }
|
| 33 |
+
|
| 34 |
+
.cols { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; }
|
| 35 |
+
@media (max-width: 900px) { .cols { grid-template-columns: 1fr; } }
|
| 36 |
+
|
| 37 |
+
label { display: block; font-size: 13px; color: var(--ink2); margin-bottom: 7px; font-weight: 600; }
|
| 38 |
+
textarea {
|
| 39 |
+
width: 100%; height: 380px; padding: 14px; border: 1px solid var(--line);
|
| 40 |
+
border-radius: 7px; background: var(--panel); color: var(--ink);
|
| 41 |
+
font: 14px/1.65 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; resize: vertical;
|
| 42 |
+
}
|
| 43 |
+
textarea:focus { outline: 2px solid var(--accent); outline-offset: -1px; }
|
| 44 |
+
|
| 45 |
+
.row { display: flex; gap: 9px; align-items: center; margin-top: 11px; flex-wrap: wrap; }
|
| 46 |
+
button {
|
| 47 |
+
font: 600 14px/1 inherit; padding: 10px 17px; border-radius: 6px;
|
| 48 |
+
border: 1px solid var(--line); background: var(--panel); color: var(--ink); cursor: pointer;
|
| 49 |
+
}
|
| 50 |
+
button.primary { background: var(--accent); border-color: var(--accent); color: #fff; }
|
| 51 |
+
button:hover { filter: brightness(1.08); }
|
| 52 |
+
.counts { color: var(--muted); font-size: 13px; margin-left: auto; }
|
| 53 |
+
|
| 54 |
+
.verdict { font-size: 16px; font-weight: 700; margin-bottom: 12px; }
|
| 55 |
+
.verdict small { font-weight: 400; color: var(--ink2); font-size: 14px; }
|
| 56 |
+
|
| 57 |
+
.finding {
|
| 58 |
+
border-left: 3px solid var(--line); background: var(--panel);
|
| 59 |
+
padding: 11px 13px; margin-bottom: 9px; border-radius: 0 6px 6px 0;
|
| 60 |
+
}
|
| 61 |
+
.finding.error { border-left-color: var(--error); }
|
| 62 |
+
.finding.warn { border-left-color: var(--warn); }
|
| 63 |
+
.tag {
|
| 64 |
+
font-size: 11px; font-weight: 700; text-transform: uppercase; letter-spacing: 0.05em;
|
| 65 |
+
}
|
| 66 |
+
.error .tag { color: var(--error); }
|
| 67 |
+
.warn .tag { color: var(--warn); }
|
| 68 |
+
.excerpt {
|
| 69 |
+
font: 13px/1.5 ui-monospace, Menlo, Consolas, monospace;
|
| 70 |
+
margin: 5px 0; color: var(--ink);
|
| 71 |
+
}
|
| 72 |
+
.fix { font-size: 13px; color: var(--ink2); }
|
| 73 |
+
.clean {
|
| 74 |
+
padding: 16px; border-radius: 7px; background: var(--panel);
|
| 75 |
+
border-left: 3px solid var(--ok); font-size: 14px; color: var(--ink2);
|
| 76 |
+
}
|
| 77 |
+
.clean b { color: var(--ink); }
|
| 78 |
+
|
| 79 |
+
details { margin-top: 18px; border-top: 1px solid var(--line); padding-top: 14px; }
|
| 80 |
+
summary { cursor: pointer; font-size: 13px; font-weight: 600; color: var(--ink2); }
|
| 81 |
+
table { width: 100%; border-collapse: collapse; font-size: 13px; margin-top: 11px; }
|
| 82 |
+
th { text-align: left; color: var(--muted); font-weight: 600; padding: 5px 8px; border-bottom: 1px solid var(--line); }
|
| 83 |
+
td { padding: 5px 8px; border-bottom: 1px solid var(--line); }
|
| 84 |
+
td.num { text-align: right; font-family: ui-monospace, Menlo, monospace; }
|
| 85 |
+
.fires { color: var(--error); font-weight: 700; }
|
| 86 |
+
.passes { color: var(--ok); }
|
| 87 |
+
.na { color: var(--muted); }
|
| 88 |
+
footer { margin-top: 34px; padding-top: 16px; border-top: 1px solid var(--line); color: var(--muted); font-size: 13px; }
|
| 89 |
+
footer a { color: var(--accent); text-decoration: none; }
|
| 90 |
+
</style>
|
| 91 |
+
</head>
|
| 92 |
+
<body>
|
| 93 |
+
<div class="wrap">
|
| 94 |
+
<header>
|
| 95 |
+
<h1>bluepencil</h1>
|
| 96 |
+
<p class="lede">A quality gate for prose. It fails <b>46.9%</b> of machine-written documents and <b>6.3%</b> of human-written ones, and both numbers were measured rather than picked.</p>
|
| 97 |
+
<p class="meta" id="calib"></p>
|
| 98 |
+
</header>
|
| 99 |
+
|
| 100 |
+
<div class="cols">
|
| 101 |
+
<div>
|
| 102 |
+
<label for="input">Your prose</label>
|
| 103 |
+
<textarea id="input" placeholder="Paste a draft. Markdown is fine; code blocks, links, tables, and blockquotes are skipped."></textarea>
|
| 104 |
+
<div class="row">
|
| 105 |
+
<button class="primary" id="run">Check</button>
|
| 106 |
+
<button id="ex-machine">Machine-written example</button>
|
| 107 |
+
<button id="ex-human">Human example</button>
|
| 108 |
+
<button id="clear">Clear</button>
|
| 109 |
+
<span class="counts" id="counts"></span>
|
| 110 |
+
</div>
|
| 111 |
+
</div>
|
| 112 |
+
<div>
|
| 113 |
+
<label>Findings</label>
|
| 114 |
+
<div id="out"><p style="color:var(--muted);font-size:14px;">Paste something and press Check. Everything runs in your browser; nothing is uploaded.</p></div>
|
| 115 |
+
<details id="cad-wrap">
|
| 116 |
+
<summary>Cadence measurements against their calibrated thresholds</summary>
|
| 117 |
+
<div id="cadence"></div>
|
| 118 |
+
</details>
|
| 119 |
+
</div>
|
| 120 |
+
</div>
|
| 121 |
+
|
| 122 |
+
<footer>
|
| 123 |
+
<p><b style="color:var(--ink)">What this will not tell you.</b> Whether the text is true, whether the argument holds, or who wrote it. It measures the surface. A document can pass every gate and still be empty, and a good writer will sometimes trip three gates deliberately.</p>
|
| 124 |
+
<p>It is a gate, not a classifier. Failing means the text contains constructions this standard rejects, which is true of plenty of human writing, as the 6.3% says. Nine of the eighteen gates fire more often on human writing than machine writing on the validation corpus; that corpus was collected around December 2022 and cannot contain constructions that only became common later.</p>
|
| 125 |
+
<p><a href="https://github.com/wolfvswhale/bluepencil">Source and CLI</a> · <a href="https://github.com/wolfvswhale/prose-eval">The harness behind the thresholds</a> · <a href="https://huggingface.co/datasets/wolfvswhale/prose-cadence-stats">The measurements</a> · <a href="https://wolfvswhale.github.io">Write-ups</a></p>
|
| 126 |
+
</footer>
|
| 127 |
+
</div>
|
| 128 |
+
|
| 129 |
+
<script type="module">
|
| 130 |
+
import { check } from "./bluepencil.js";
|
| 131 |
+
|
| 132 |
+
const MACHINE = `In today's fast-paced world, technical documentation is not just a nice-to-have, it's a competitive advantage. It's important to note that teams who invest in clear writing consistently outperform their peers. Studies show a strong correlation between documentation quality and delivery speed.
|
| 133 |
+
|
| 134 |
+
Our approach stands as a testament to this principle. We built a system that is fast, reliable, and maintainable. Adoption tripled in the first quarter, highlighting the strength of the underlying model. The platform serves as a foundation for everything the team builds.
|
| 135 |
+
|
| 136 |
+
Moreover, the framework scales effortlessly. Furthermore, onboarding time dropped substantially. Additionally, support tickets decreased across every category we measured.
|
| 137 |
+
|
| 138 |
+
What makes this approach different is that it represents a fundamental shift in how teams think about knowledge. Documentation is no longer an afterthought, it is a core deliverable. Experts agree that this reframing is essential for organizations navigating the complexity of modern software delivery.
|
| 139 |
+
|
| 140 |
+
In conclusion, good documentation pays for itself many times over.`;
|
| 141 |
+
|
| 142 |
+
const HUMAN = `The pump failed on a Tuesday. We pulled the housing and found the seal had cracked along one edge, most likely from the freeze in January that nobody thought to check for afterward.
|
| 143 |
+
|
| 144 |
+
Replacing it took an hour.
|
| 145 |
+
|
| 146 |
+
It has run clean since, though I want to check the pressure again after a month of real use, because a seal that fails once in a specific place tends to fail there again. That is the part the manual does not tell you. You learn it by having the same thing break twice and finally looking at where.
|
| 147 |
+
|
| 148 |
+
There is a second seal on the intake side that I have not touched. It looks fine. But the manual says both were replaced at the same time by whoever owned the place before us, which means both have the same hours on them, and one has already gone. So I ordered two.
|
| 149 |
+
|
| 150 |
+
We will know by March.`;
|
| 151 |
+
|
| 152 |
+
const esc = (s) => s.replace(/[&<>]/g, (c) => ({ "&": "&", "<": "<", ">": ">" }[c]));
|
| 153 |
+
const spec = await fetch("./gates.json").then((r) => r.json());
|
| 154 |
+
|
| 155 |
+
const m = spec.meta || {};
|
| 156 |
+
document.getElementById("calib").innerHTML =
|
| 157 |
+
`Every cadence threshold was set by measuring that statistic across <b style="color:var(--ink)">${(m.n_documents || 0).toLocaleString()} human-written documents</b> ` +
|
| 158 |
+
`in three registers, then placed at the percentile that yields a <b style="color:var(--ink)">${Math.round((m.target_fpr || 0.05) * 100)}% false-positive rate</b>. ` +
|
| 159 |
+
`That rate is a property of the tool, not a hope.`;
|
| 160 |
+
|
| 161 |
+
const $ = (id) => document.getElementById(id);
|
| 162 |
+
|
| 163 |
+
function render(result) {
|
| 164 |
+
const out = $("out");
|
| 165 |
+
if (!result.findings.length) {
|
| 166 |
+
out.innerHTML = `<div class="clean"><b>Clean.</b> No gate fired. That is not a claim the writing is good, only that it does not contain the constructions this standard rejects.</div>`;
|
| 167 |
+
} else {
|
| 168 |
+
const verdict = result.errors ? "FAILS" : "passes with warnings";
|
| 169 |
+
const colour = result.errors ? "var(--error)" : "var(--warn)";
|
| 170 |
+
let html = `<div class="verdict" style="color:${colour}">${verdict} <small>${result.errors} error(s), ${result.warnings} warning(s) across ${result.words} words</small></div>`;
|
| 171 |
+
for (const f of result.findings) {
|
| 172 |
+
const where = f.line ? `line ${f.line}` : "whole document";
|
| 173 |
+
const body = f.excerpt
|
| 174 |
+
? `<div class="excerpt">“${esc(f.excerpt)}”</div>`
|
| 175 |
+
: `<div class="excerpt" style="font-family:inherit">${esc(f.message)}</div>`;
|
| 176 |
+
html += `<div class="finding ${f.severity}">
|
| 177 |
+
<div class="tag">${f.severity} · ${esc(f.gate)} · ${where}</div>
|
| 178 |
+
${body}<div class="fix">→ ${esc(f.suggestion)}</div></div>`;
|
| 179 |
+
}
|
| 180 |
+
out.innerHTML = html;
|
| 181 |
+
}
|
| 182 |
+
|
| 183 |
+
let rows = "";
|
| 184 |
+
for (const c of result.cadence) {
|
| 185 |
+
if (c.threshold === null) {
|
| 186 |
+
rows += `<tr><td>${c.name}</td><td colspan="3" class="na">disabled — could not be calibrated</td></tr>`;
|
| 187 |
+
} else if (!c.measurable) {
|
| 188 |
+
rows += `<tr><td>${c.name}</td><td colspan="3" class="na">not measurable on this text</td></tr>`;
|
| 189 |
+
} else {
|
| 190 |
+
rows += `<tr><td>${c.name}</td><td class="num">${c.value.toFixed(3)}</td>
|
| 191 |
+
<td class="num na">${c.direction === "low" ? "<" : ">"} ${c.threshold.toFixed(3)}</td>
|
| 192 |
+
<td class="${c.fires ? "fires" : "passes"}">${c.fires ? "fires" : "ok"}</td></tr>`;
|
| 193 |
+
}
|
| 194 |
+
}
|
| 195 |
+
$("cadence").innerHTML = `<table><thead><tr><th>gate</th><th style="text-align:right">measured</th><th style="text-align:right">threshold</th><th></th></tr></thead><tbody>${rows}</tbody></table>`;
|
| 196 |
+
$("counts").textContent = `${result.words} words · ${result.sentences} sentences · ${result.paragraphs} paragraphs`;
|
| 197 |
+
}
|
| 198 |
+
|
| 199 |
+
function run() {
|
| 200 |
+
const text = $("input").value;
|
| 201 |
+
if (!text.trim()) {
|
| 202 |
+
$("out").innerHTML = `<p style="color:var(--muted);font-size:14px;">Paste some prose above.</p>`;
|
| 203 |
+
$("counts").textContent = "";
|
| 204 |
+
return;
|
| 205 |
+
}
|
| 206 |
+
render(check(text, spec));
|
| 207 |
+
}
|
| 208 |
+
|
| 209 |
+
$("run").onclick = run;
|
| 210 |
+
$("ex-machine").onclick = () => { $("input").value = MACHINE; $("cad-wrap").open = true; run(); };
|
| 211 |
+
$("ex-human").onclick = () => { $("input").value = HUMAN; $("cad-wrap").open = true; run(); };
|
| 212 |
+
$("clear").onclick = () => { $("input").value = ""; run(); };
|
| 213 |
+
$("input").addEventListener("keydown", (e) => {
|
| 214 |
+
if ((e.metaKey || e.ctrlKey) && e.key === "Enter") run();
|
| 215 |
+
});
|
| 216 |
+
</script>
|
| 217 |
+
</body>
|
| 218 |
</html>
|