wolfvswhale commited on
Commit
ed5252d
·
verified ·
1 Parent(s): 6f235fb

bluepencil: a calibrated quality gate for prose, running entirely in the browser

Browse files
Files changed (4) hide show
  1. README.md +49 -5
  2. bluepencil.js +210 -0
  3. gates.json +233 -0
  4. index.html +216 -17
README.md CHANGED
@@ -1,10 +1,54 @@
1
  ---
2
- title: Bluepencil
3
- emoji: 📊
4
- colorFrom: blue
5
- colorTo: pink
6
  sdk: static
 
7
  pinned: false
 
 
 
 
 
 
 
 
 
8
  ---
9
 
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: bluepencil
3
+ emoji: ✏️
4
+ colorFrom: indigo
5
+ colorTo: gray
6
  sdk: static
7
+ app_file: index.html
8
  pinned: false
9
+ license: mit
10
+ short_description: A prose quality gate with a measured 5% false-positive rate
11
+ tags:
12
+ - text-classification
13
+ - writing
14
+ - ai-detection
15
+ - evaluation
16
+ - linter
17
+ - prose
18
  ---
19
 
20
+ # bluepencil
21
+
22
+ A quality gate for prose. Paste a draft, see which gates fire and why.
23
+
24
+ It fails 46.9% of machine-written documents and 6.3% of human-written ones. Both numbers were measured on 1,500 held-out documents per class rather than asserted.
25
+
26
+ ## What makes it different from the other detectors on the Hub
27
+
28
+ Most tools of this kind give you a percentage and no way to check it. This one reports named findings with line numbers and a fix for each, and every threshold behind those findings was calibrated rather than chosen.
29
+
30
+ The method: measure each statistic across 2,602 human-written documents spanning three registers, informal forum answers, formal scientific abstracts, and multi-paragraph encyclopedia articles, then place the threshold at the percentile that produces a 5% false-positive rate. The panel in the app shows you the measured value against the threshold for every cadence gate, so you can disagree with it on the evidence.
31
+
32
+ Eighteen gates in two families. Pattern gates point at a span of text. Cadence gates measure the whole document, and those are the ones that catch prose which had every flagged word swapped out and still reads flat.
33
+
34
+ Everything runs in your browser. Nothing you paste is uploaded anywhere.
35
+
36
+ ## What it will not tell you
37
+
38
+ Whether the text is true, whether the argument holds, or who wrote it. It measures the surface. A document can pass every gate and still be empty.
39
+
40
+ It is a gate, not a classifier. Failing means the text contains constructions a particular editorial standard rejects, which is true of plenty of good human writing, as the 6.3% says.
41
+
42
+ Nine of the eighteen gates fire more often on human writing than machine writing on the validation corpus. That is published rather than hidden, along with the reason: the validation corpus was collected around December 2022 and cannot contain constructions that only became common later.
43
+
44
+ ## Elsewhere
45
+
46
+ The CLI, the pre-commit hook, the GitHub Action, and the agent skill: [github.com/wolfvswhale/bluepencil](https://github.com/wolfvswhale/bluepencil)
47
+
48
+ The evaluation harness that produced the thresholds, including its own negative results: [github.com/wolfvswhale/prose-eval](https://github.com/wolfvswhale/prose-eval)
49
+
50
+ The measurements underneath it all: [prose-cadence-stats](https://huggingface.co/datasets/wolfvswhale/prose-cadence-stats)
51
+
52
+ Rules derived from Wikipedia's [Signs of AI writing](https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing), maintained by WikiProject AI Cleanup, plus a house editing standard.
53
+
54
+ Built by J. Alderman Lyell. MIT licensed.
bluepencil.js ADDED
@@ -0,0 +1,210 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ /* bluepencil, ported to run in the browser.
2
+ *
3
+ * The Python package is the reference implementation. This is a faithful port
4
+ * of the scoring logic so the Space can be static, which on Hugging Face is
5
+ * the only tier that is free. It also means nothing you paste leaves your
6
+ * machine, and there is no cold start.
7
+ *
8
+ * Gate definitions and every calibrated threshold are loaded from gates.json,
9
+ * which is generated from the Python package rather than retyped, so the two
10
+ * cannot drift.
11
+ */
12
+
13
+ const TRAILING_WS = /[ \t]+$/;
14
+
15
+ /* ---------- markdown stripping -------------------------------------------
16
+ * Replace non-prose spans with spaces, preserving newlines and total length,
17
+ * so character offsets still map to the right line and column.
18
+ */
19
+ function blank(match) {
20
+ return match.replace(/[^\n]/g, " ");
21
+ }
22
+
23
+ function stripNonProse(text) {
24
+ let out = text;
25
+ const sub = (re) => { out = out.replace(re, blank); };
26
+
27
+ sub(/^---\r?\n[\s\S]*?\r?\n---[ \t]*\r?\n/); // YAML front matter
28
+ // fenced code: pair up fences
29
+ const fence = /^([ \t]*)(`{3,}|~{3,}).*$/gm;
30
+ const fences = [...out.matchAll(fence)];
31
+ for (let i = 0; i + 1 < fences.length; i += 2) {
32
+ const start = fences[i].index;
33
+ const end = fences[i + 1].index + fences[i + 1][0].length;
34
+ out = out.slice(0, start) + blank(out.slice(start, end)) + out.slice(end);
35
+ }
36
+ const odd = [...out.matchAll(fence)];
37
+ if (odd.length % 2 === 1) {
38
+ const s = odd[odd.length - 1].index;
39
+ out = out.slice(0, s) + blank(out.slice(s));
40
+ }
41
+
42
+ sub(/!\[[^\]]*\]\([^)]*\)/g); // images
43
+ sub(/\]\([^)\s]+(?:\s+"[^"]*")?\)/g); // link targets
44
+ sub(/`[^`\n]+`/g); // inline code
45
+ sub(/https?:\/\/\S+/g); // bare urls
46
+ sub(/<[^>\n]{1,120}>/g); // html tags
47
+ sub(/^[ \t]{0,3}\|.*\|[ \t]*$/gm); // table rows
48
+ sub(/^[ \t]{0,3}#{1,6}[ \t]+.*$/gm); // headings
49
+ sub(/^[ \t]{0,3}>.*$/gm); // blockquotes
50
+ return out;
51
+ }
52
+
53
+ /* ---------- parsing ------------------------------------------------------ */
54
+
55
+ const WORD_RE = /[A-Za-z0-9][A-Za-z0-9'’-]*/g;
56
+ const SENT_SPLIT = /(?<=[.!?])["')\]]*\s+/;
57
+
58
+ function words(s) { return s.match(WORD_RE) || []; }
59
+
60
+ function parse(raw) {
61
+ const text = stripNonProse(raw);
62
+ const lineStarts = [0];
63
+ for (let i = 0; i < raw.length; i++) if (raw[i] === "\n") lineStarts.push(i + 1);
64
+
65
+ const paragraphs = [];
66
+ let offset = 0;
67
+ for (const chunk of text.split(/\n[ \t]*\n/)) {
68
+ const trimmed = chunk.trim();
69
+ if (trimmed) {
70
+ const sentences = [];
71
+ let cursor = 0;
72
+ for (const piece of chunk.split(SENT_SPLIT)) {
73
+ const t = piece.trim();
74
+ if (!t) continue;
75
+ let idx = chunk.indexOf(t, cursor);
76
+ if (idx === -1) idx = cursor;
77
+ const w = words(t);
78
+ if (w.length) sentences.push({ text: t, start: offset + idx, words: w });
79
+ cursor = idx + t.length;
80
+ }
81
+ paragraphs.push({ text: trimmed, sentences });
82
+ }
83
+ offset += chunk.length + 2;
84
+ }
85
+
86
+ const sentences = paragraphs.flatMap((p) => p.sentences);
87
+ return { raw, text, lineStarts, paragraphs, sentences, words: words(text) };
88
+ }
89
+
90
+ function position(doc, offset) {
91
+ let lo = 0, hi = doc.lineStarts.length - 1;
92
+ while (lo < hi) {
93
+ const mid = Math.ceil((lo + hi) / 2);
94
+ if (doc.lineStarts[mid] <= offset) lo = mid; else hi = mid - 1;
95
+ }
96
+ return { line: lo + 1, column: offset - doc.lineStarts[lo] + 1 };
97
+ }
98
+
99
+ /* ---------- statistics --------------------------------------------------- */
100
+
101
+ const mean = (a) => a.reduce((x, y) => x + y, 0) / a.length;
102
+ function pstdev(a) {
103
+ const m = mean(a);
104
+ return Math.sqrt(a.reduce((s, v) => s + (v - m) ** 2, 0) / a.length);
105
+ }
106
+ function cv(a) {
107
+ if (a.length < 2) return 0;
108
+ const m = mean(a);
109
+ return m === 0 ? 0 : pstdev(a) / m;
110
+ }
111
+
112
+ const MEASURES = {
113
+ "flat-cadence": (d) => {
114
+ const l = d.sentences.map((s) => s.words.length);
115
+ return l.length < 4 ? null : cv(l);
116
+ },
117
+ "uniform-paragraphs": (d) => {
118
+ const c = d.paragraphs.filter((p) => p.sentences.length).map((p) => p.sentences.length);
119
+ return c.length < 3 ? null : cv(c);
120
+ },
121
+ "no-short-sentences": (d) => {
122
+ const l = d.sentences.map((s) => s.words.length);
123
+ return l.length < 4 ? null : l.filter((n) => n <= 8).length / l.length;
124
+ },
125
+ "triplet-habit": (d) => {
126
+ if (d.words.length < 60) return null;
127
+ const m = d.text.match(/\b[\w'-]+,\s+[\w'-]+,\s+(?:and|or)\s+[\w'-]+/g) || [];
128
+ return (m.length * 1000) / d.words.length;
129
+ },
130
+ "em-dash-habit": (d) => {
131
+ if (d.words.length < 60) return null;
132
+ return ((d.text.split("—").length - 1) * 1000) / d.words.length;
133
+ },
134
+ "repeated-openers": (d) => {
135
+ const starts = d.sentences.filter((s) => s.words.length).map((s) => s.words[0].toLowerCase());
136
+ if (starts.length < 5) return null;
137
+ let r = 0;
138
+ for (let i = 1; i < starts.length; i++) if (starts[i] === starts[i - 1]) r++;
139
+ return r / (starts.length - 1);
140
+ },
141
+ };
142
+
143
+ /* ---------- the check ---------------------------------------------------- */
144
+
145
+ export function check(raw, spec, minWords = 120) {
146
+ const doc = parse(raw);
147
+ const findings = [];
148
+
149
+ for (const gate of spec.patterns) {
150
+ if (gate.severity === "off") continue;
151
+ let hits = [];
152
+ for (const src of gate.patterns) {
153
+ const re = new RegExp(src, "gi");
154
+ for (const m of doc.text.matchAll(re)) hits.push({ start: m.index, span: m[0] });
155
+ }
156
+ if (!hits.length) continue;
157
+
158
+ // de-duplicate overlapping matches from sibling patterns
159
+ hits.sort((a, b) => a.start - b.start);
160
+ const kept = [];
161
+ let lastEnd = -1;
162
+ for (const h of hits) {
163
+ if (h.start >= lastEnd) { kept.push(h); lastEnd = h.start + h.span.length; }
164
+ }
165
+
166
+ if (gate.budget > 0 && doc.words.length) {
167
+ const allowed = Math.floor((gate.budget * doc.words.length) / 1000);
168
+ if (kept.length <= allowed) continue;
169
+ }
170
+
171
+ for (const h of kept) {
172
+ const { line, column } = position(doc, h.start);
173
+ findings.push({
174
+ gate: gate.name, severity: gate.severity, kind: "pattern",
175
+ message: gate.description, suggestion: gate.suggestion,
176
+ excerpt: h.span.replace(/\s+/g, " ").slice(0, 90),
177
+ line, column,
178
+ });
179
+ }
180
+ }
181
+
182
+ const cadence = [];
183
+ for (const gate of spec.cadence) {
184
+ const measure = MEASURES[gate.name];
185
+ const value = doc.words.length >= Math.max(gate.min_words, minWords) && measure ? measure(doc) : null;
186
+ const fires = gate.threshold !== null && value !== null &&
187
+ (gate.direction === "low" ? value < gate.threshold : value > gate.threshold);
188
+ cadence.push({ ...gate, value, fires, measurable: value !== null });
189
+ if (fires && gate.severity !== "off") {
190
+ const cmp = gate.direction === "low" ? "below" : "above";
191
+ findings.push({
192
+ gate: gate.name, severity: gate.severity, kind: "cadence",
193
+ message: `${gate.description} Measured ${value.toFixed(3)}, ${cmp} the calibrated threshold of ${gate.threshold.toFixed(3)}.`,
194
+ suggestion: gate.suggestion, excerpt: "", line: 0, column: 0,
195
+ });
196
+ }
197
+ }
198
+
199
+ const rank = { error: 2, warn: 1, off: 0 };
200
+ findings.sort((a, b) => (rank[b.severity] - rank[a.severity]) || (a.line - b.line) || (a.column - b.column));
201
+
202
+ return {
203
+ findings, cadence,
204
+ words: doc.words.length,
205
+ sentences: doc.sentences.length,
206
+ paragraphs: doc.paragraphs.length,
207
+ errors: findings.filter((f) => f.severity === "error").length,
208
+ warnings: findings.filter((f) => f.severity === "warn").length,
209
+ };
210
+ }
gates.json ADDED
@@ -0,0 +1,233 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "patterns": [
3
+ {
4
+ "name": "negative-parallelism",
5
+ "description": "'Not just X, it's Y' and its relatives. The most recognizable machine construction.",
6
+ "suggestion": "State the correct thing. Leave the wrong thing out.",
7
+ "severity": "error",
8
+ "budget": 0.0,
9
+ "patterns": [
10
+ "not just [^.!?]{1,50}?,? (?:but |it'?s |it is )",
11
+ "not only\\b[^.!?]{0,60}?\\bbut also\\b",
12
+ "(?:isn'?t|is not) (?:about|just)\\b[^.!?]{0,50}?[;,] it'?s",
13
+ "it'?s not\\b[^.!?]{0,40}?[;,] it'?s\\b"
14
+ ]
15
+ },
16
+ {
17
+ "name": "inflated-significance",
18
+ "description": "Announcing that something matters instead of showing what it did.",
19
+ "suggestion": "Show what it did, with a fact. Cut the claim of importance.",
20
+ "severity": "error",
21
+ "budget": 0.0,
22
+ "patterns": [
23
+ "stands? as a testament",
24
+ "(?:is|are) a testament to",
25
+ "plays? a (?:vital|pivotal|crucial|key|significant) role",
26
+ "underscor\\w+ (?:its|the|their) (?:importance|significance)",
27
+ "leav\\w+ a lasting (?:impact|impression|legacy)",
28
+ "watershed moment",
29
+ "cannot be overstated",
30
+ "solidif\\w+ (?:its|their) (?:position|place|status)"
31
+ ]
32
+ },
33
+ {
34
+ "name": "participle-tail",
35
+ "description": "A clause tacked onto a sentence to fake analysis.",
36
+ "suggestion": "Explain the consequence concretely, or stop at the fact.",
37
+ "severity": "error",
38
+ "budget": 0.0,
39
+ "patterns": [
40
+ ",\\s+(?:highlighting|underscoring|illustrating|reflecting|emphasizing|showcasing|paving|cementing|signaling|marking)\\b[^.!?]{0,80}[.!?]"
41
+ ]
42
+ },
43
+ {
44
+ "name": "editorializing",
45
+ "description": "Telling the reader that something is worth noting instead of noting it.",
46
+ "suggestion": "Just write the note.",
47
+ "severity": "error",
48
+ "budget": 0.0,
49
+ "patterns": [
50
+ "it'?s important to (?:note|remember|understand|consider)",
51
+ "it is important to (?:note|remember|understand|consider)",
52
+ "it'?s worth (?:noting|remembering|mentioning)",
53
+ "no discussion would be complete without",
54
+ "needless to say"
55
+ ]
56
+ },
57
+ {
58
+ "name": "hollow-conclusion",
59
+ "description": "Restating a passage the reader just finished.",
60
+ "suggestion": "End on the last real thing there is to say. A piece can just stop.",
61
+ "severity": "error",
62
+ "budget": 0.0,
63
+ "patterns": [
64
+ "\\bin (?:summary|conclusion)\\b",
65
+ "(?:^|\\n)\\s*overall,",
66
+ "\\bat the end of the day\\b",
67
+ "\\bto sum up\\b"
68
+ ]
69
+ },
70
+ {
71
+ "name": "throat-clearing",
72
+ "description": "Warming up instead of starting.",
73
+ "suggestion": "Delete it. Start with the first real sentence.",
74
+ "severity": "error",
75
+ "budget": 0.0,
76
+ "patterns": [
77
+ "in today'?s (?:fast-paced|modern|digital|ever-changing) world",
78
+ "in the (?:ever-)?(?:evolving|changing) (?:landscape|world) of",
79
+ "let'?s (?:dive in|unpack|explore this)",
80
+ "here'?s a comprehensive"
81
+ ]
82
+ },
83
+ {
84
+ "name": "engagement-bait",
85
+ "description": "Nudging the reader instead of informing them.",
86
+ "suggestion": "Cut it.",
87
+ "severity": "error",
88
+ "budget": 0.0,
89
+ "patterns": [
90
+ "let that sink in",
91
+ "read that again",
92
+ "this changes everything",
93
+ "here'?s the thing\\b"
94
+ ]
95
+ },
96
+ {
97
+ "name": "vague-attribution",
98
+ "description": "Claims with no named source.",
99
+ "suggestion": "Name the person, paper, or outlet with a date, or drop the claim.",
100
+ "severity": "error",
101
+ "budget": 0.0,
102
+ "patterns": [
103
+ "\\bstudies (?:show|suggest|indicate)\\b",
104
+ "\\bexperts (?:say|agree|note)\\b",
105
+ "\\bresearch (?:shows|suggests|indicates)\\b",
106
+ "\\b(?:some|many) (?:critics|observers|analysts) (?:argue|note|suggest)\\b",
107
+ "\\bindustry reports suggest\\b"
108
+ ]
109
+ },
110
+ {
111
+ "name": "promotional",
112
+ "description": "Brochure register.",
113
+ "suggestion": "Replace the adjective with a fact, or cut it.",
114
+ "severity": "warn",
115
+ "budget": 0.0,
116
+ "patterns": [
117
+ "rich (?:cultural )?(?:heritage|history|tapestry)",
118
+ "stunning natural beauty",
119
+ "nestled in the heart of",
120
+ "must-(?:visit|see|read|have)\\b",
121
+ "\\bbreathtaking\\b",
122
+ "\\bworld-class\\b"
123
+ ]
124
+ },
125
+ {
126
+ "name": "transition-crutch",
127
+ "description": "Connective tissue that can usually be deleted with no loss.",
128
+ "suggestion": "Delete it, or use a plain connector: but, so, then.",
129
+ "severity": "warn",
130
+ "budget": 4.081,
131
+ "patterns": [
132
+ "\\bmoreover\\b",
133
+ "\\bfurthermore\\b",
134
+ "\\badditionally\\b",
135
+ "\\bon top of that\\b",
136
+ "\\bthat said\\b"
137
+ ]
138
+ },
139
+ {
140
+ "name": "copula-avoidance",
141
+ "description": "Swapping 'is' for something weightier.",
142
+ "suggestion": "Use 'is' or 'was'.",
143
+ "severity": "warn",
144
+ "budget": 0.0,
145
+ "patterns": [
146
+ "\\bserves? as\\b",
147
+ "\\bstands? as\\b",
148
+ "\\brepresents? a\\b"
149
+ ]
150
+ },
151
+ {
152
+ "name": "assistant-residue",
153
+ "description": "Chat-assistant phrasing left in published text.",
154
+ "suggestion": "Delete. This should never reach a reader.",
155
+ "severity": "error",
156
+ "budget": 0.0,
157
+ "patterns": [
158
+ "\\bas an AI language model\\b",
159
+ "\\bI hope this helps\\b",
160
+ "\\blet me know if you (?:need|have)\\b",
161
+ "\\bfeel free to reach out\\b",
162
+ "\\bas of my (?:last )?knowledge (?:update|cutoff)\\b",
163
+ "\\bI hope this (?:message|email) finds you well\\b"
164
+ ]
165
+ }
166
+ ],
167
+ "cadence": [
168
+ {
169
+ "name": "flat-cadence",
170
+ "description": "Sentence lengths vary less than human writing typically does.",
171
+ "suggestion": "Rewrite one sentence in every three-sentence run to a different length. A short sentence after two long ones does more than any word swap.",
172
+ "severity": "warn",
173
+ "direction": "low",
174
+ "threshold": 0.2367,
175
+ "min_words": 120
176
+ },
177
+ {
178
+ "name": "uniform-paragraphs",
179
+ "description": "Paragraphs are all about the same length.",
180
+ "suggestion": "Break one paragraph early and let another run long. A one-line paragraph is allowed.",
181
+ "severity": "warn",
182
+ "direction": "low",
183
+ "threshold": 0.5227,
184
+ "min_words": 120
185
+ },
186
+ {
187
+ "name": "no-short-sentences",
188
+ "description": "Nothing in the draft is short.",
189
+ "suggestion": "Cut one sentence down to under eight words. It changes the rhythm of everything around it.",
190
+ "severity": "warn",
191
+ "direction": "low",
192
+ "threshold": null,
193
+ "min_words": 120
194
+ },
195
+ {
196
+ "name": "triplet-habit",
197
+ "description": "Reflexive lists of three: 'x, y, and z' over and over.",
198
+ "suggestion": "Use the number of items that exists. Two is fine. One is often best.",
199
+ "severity": "warn",
200
+ "direction": "high",
201
+ "threshold": 5.1787,
202
+ "min_words": 120
203
+ },
204
+ {
205
+ "name": "em-dash-habit",
206
+ "description": "Em dashes used as a default connector.",
207
+ "suggestion": "Use a comma, a colon, parentheses, or two sentences.",
208
+ "severity": "warn",
209
+ "direction": "high",
210
+ "threshold": 1.4888,
211
+ "min_words": 120
212
+ },
213
+ {
214
+ "name": "repeated-openers",
215
+ "description": "Consecutive sentences starting with the same word.",
216
+ "suggestion": "Vary the opener, unless the repetition is deliberate.",
217
+ "severity": "warn",
218
+ "direction": "high",
219
+ "threshold": 0.2222,
220
+ "min_words": 120
221
+ }
222
+ ],
223
+ "meta": {
224
+ "calibrated": true,
225
+ "corpus": "HC3 human answers (informal) + human scientific abstracts (formal) + wikitext-103 articles (multi-paragraph)",
226
+ "n_documents": 2602,
227
+ "informal": 1000,
228
+ "formal": 1000,
229
+ "structured": 602,
230
+ "min_words": 120,
231
+ "target_fpr": 0.05
232
+ }
233
+ }
index.html CHANGED
@@ -1,19 +1,218 @@
1
  <!doctype html>
2
- <html>
3
- <head>
4
- <meta charset="utf-8" />
5
- <meta name="viewport" content="width=device-width" />
6
- <title>My static Space</title>
7
- <link rel="stylesheet" href="style.css" />
8
- </head>
9
- <body>
10
- <div class="card">
11
- <h1>Welcome to your static Space!</h1>
12
- <p>You can modify this app directly by editing <i>index.html</i> in the Files and versions tab.</p>
13
- <p>
14
- Also don't forget to check the
15
- <a href="https://huggingface.co/docs/hub/spaces" target="_blank">Spaces documentation</a>.
16
- </p>
17
- </div>
18
- </body>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
  </html>
 
1
  <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1">
6
+ <title>bluepencil a calibrated quality gate for prose</title>
7
+ <style>
8
+ :root {
9
+ --surface: #fcfcfb; --panel: #ffffff; --line: #e6e6e0;
10
+ --ink: #0b0b0b; --ink2: #52514e; --muted: #8a8a85;
11
+ --error: #b4232a; --warn: #a06010; --ok: #2e7d32; --accent: #2a78d6;
12
+ }
13
+ @media (prefers-color-scheme: dark) {
14
+ :root {
15
+ --surface: #16161a; --panel: #1d1d21; --line: #2e2e34;
16
+ --ink: #f2f2ef; --ink2: #b8b7b1; --muted: #7e7e86;
17
+ --error: #ff6b6b; --warn: #e0a34a; --ok: #6cc46c; --accent: #6aa9f0;
18
+ }
19
+ }
20
+ * { box-sizing: border-box; }
21
+ body {
22
+ margin: 0; background: var(--surface); color: var(--ink);
23
+ font: 15px/1.6 ui-sans-serif, -apple-system, "Segoe UI", Roboto, Helvetica, Arial, sans-serif;
24
+ }
25
+ .wrap { max-width: 1180px; margin: 0 auto; padding: 26px 20px 60px; }
26
+ header { border-bottom: 2px solid var(--ink); padding-bottom: 14px; margin-bottom: 20px; }
27
+ h1 { font-size: 25px; margin: 0 0 6px; letter-spacing: -0.02em; }
28
+ .lede { color: var(--ink2); font-size: 15px; margin: 0; max-width: 74ch; }
29
+ .lede b { color: var(--ink); }
30
+ .meta { color: var(--muted); font-size: 13px; margin-top: 9px; }
31
+ .meta a { color: var(--accent); text-decoration: none; }
32
+ .meta a:hover { text-decoration: underline; }
33
+
34
+ .cols { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; }
35
+ @media (max-width: 900px) { .cols { grid-template-columns: 1fr; } }
36
+
37
+ label { display: block; font-size: 13px; color: var(--ink2); margin-bottom: 7px; font-weight: 600; }
38
+ textarea {
39
+ width: 100%; height: 380px; padding: 14px; border: 1px solid var(--line);
40
+ border-radius: 7px; background: var(--panel); color: var(--ink);
41
+ font: 14px/1.65 ui-monospace, SFMono-Regular, Menlo, Consolas, monospace; resize: vertical;
42
+ }
43
+ textarea:focus { outline: 2px solid var(--accent); outline-offset: -1px; }
44
+
45
+ .row { display: flex; gap: 9px; align-items: center; margin-top: 11px; flex-wrap: wrap; }
46
+ button {
47
+ font: 600 14px/1 inherit; padding: 10px 17px; border-radius: 6px;
48
+ border: 1px solid var(--line); background: var(--panel); color: var(--ink); cursor: pointer;
49
+ }
50
+ button.primary { background: var(--accent); border-color: var(--accent); color: #fff; }
51
+ button:hover { filter: brightness(1.08); }
52
+ .counts { color: var(--muted); font-size: 13px; margin-left: auto; }
53
+
54
+ .verdict { font-size: 16px; font-weight: 700; margin-bottom: 12px; }
55
+ .verdict small { font-weight: 400; color: var(--ink2); font-size: 14px; }
56
+
57
+ .finding {
58
+ border-left: 3px solid var(--line); background: var(--panel);
59
+ padding: 11px 13px; margin-bottom: 9px; border-radius: 0 6px 6px 0;
60
+ }
61
+ .finding.error { border-left-color: var(--error); }
62
+ .finding.warn { border-left-color: var(--warn); }
63
+ .tag {
64
+ font-size: 11px; font-weight: 700; text-transform: uppercase; letter-spacing: 0.05em;
65
+ }
66
+ .error .tag { color: var(--error); }
67
+ .warn .tag { color: var(--warn); }
68
+ .excerpt {
69
+ font: 13px/1.5 ui-monospace, Menlo, Consolas, monospace;
70
+ margin: 5px 0; color: var(--ink);
71
+ }
72
+ .fix { font-size: 13px; color: var(--ink2); }
73
+ .clean {
74
+ padding: 16px; border-radius: 7px; background: var(--panel);
75
+ border-left: 3px solid var(--ok); font-size: 14px; color: var(--ink2);
76
+ }
77
+ .clean b { color: var(--ink); }
78
+
79
+ details { margin-top: 18px; border-top: 1px solid var(--line); padding-top: 14px; }
80
+ summary { cursor: pointer; font-size: 13px; font-weight: 600; color: var(--ink2); }
81
+ table { width: 100%; border-collapse: collapse; font-size: 13px; margin-top: 11px; }
82
+ th { text-align: left; color: var(--muted); font-weight: 600; padding: 5px 8px; border-bottom: 1px solid var(--line); }
83
+ td { padding: 5px 8px; border-bottom: 1px solid var(--line); }
84
+ td.num { text-align: right; font-family: ui-monospace, Menlo, monospace; }
85
+ .fires { color: var(--error); font-weight: 700; }
86
+ .passes { color: var(--ok); }
87
+ .na { color: var(--muted); }
88
+ footer { margin-top: 34px; padding-top: 16px; border-top: 1px solid var(--line); color: var(--muted); font-size: 13px; }
89
+ footer a { color: var(--accent); text-decoration: none; }
90
+ </style>
91
+ </head>
92
+ <body>
93
+ <div class="wrap">
94
+ <header>
95
+ <h1>bluepencil</h1>
96
+ <p class="lede">A quality gate for prose. It fails <b>46.9%</b> of machine-written documents and <b>6.3%</b> of human-written ones, and both numbers were measured rather than picked.</p>
97
+ <p class="meta" id="calib"></p>
98
+ </header>
99
+
100
+ <div class="cols">
101
+ <div>
102
+ <label for="input">Your prose</label>
103
+ <textarea id="input" placeholder="Paste a draft. Markdown is fine; code blocks, links, tables, and blockquotes are skipped."></textarea>
104
+ <div class="row">
105
+ <button class="primary" id="run">Check</button>
106
+ <button id="ex-machine">Machine-written example</button>
107
+ <button id="ex-human">Human example</button>
108
+ <button id="clear">Clear</button>
109
+ <span class="counts" id="counts"></span>
110
+ </div>
111
+ </div>
112
+ <div>
113
+ <label>Findings</label>
114
+ <div id="out"><p style="color:var(--muted);font-size:14px;">Paste something and press Check. Everything runs in your browser; nothing is uploaded.</p></div>
115
+ <details id="cad-wrap">
116
+ <summary>Cadence measurements against their calibrated thresholds</summary>
117
+ <div id="cadence"></div>
118
+ </details>
119
+ </div>
120
+ </div>
121
+
122
+ <footer>
123
+ <p><b style="color:var(--ink)">What this will not tell you.</b> Whether the text is true, whether the argument holds, or who wrote it. It measures the surface. A document can pass every gate and still be empty, and a good writer will sometimes trip three gates deliberately.</p>
124
+ <p>It is a gate, not a classifier. Failing means the text contains constructions this standard rejects, which is true of plenty of human writing, as the 6.3% says. Nine of the eighteen gates fire more often on human writing than machine writing on the validation corpus; that corpus was collected around December 2022 and cannot contain constructions that only became common later.</p>
125
+ <p><a href="https://github.com/wolfvswhale/bluepencil">Source and CLI</a> · <a href="https://github.com/wolfvswhale/prose-eval">The harness behind the thresholds</a> · <a href="https://huggingface.co/datasets/wolfvswhale/prose-cadence-stats">The measurements</a> · <a href="https://wolfvswhale.github.io">Write-ups</a></p>
126
+ </footer>
127
+ </div>
128
+
129
+ <script type="module">
130
+ import { check } from "./bluepencil.js";
131
+
132
+ const MACHINE = `In today's fast-paced world, technical documentation is not just a nice-to-have, it's a competitive advantage. It's important to note that teams who invest in clear writing consistently outperform their peers. Studies show a strong correlation between documentation quality and delivery speed.
133
+
134
+ Our approach stands as a testament to this principle. We built a system that is fast, reliable, and maintainable. Adoption tripled in the first quarter, highlighting the strength of the underlying model. The platform serves as a foundation for everything the team builds.
135
+
136
+ Moreover, the framework scales effortlessly. Furthermore, onboarding time dropped substantially. Additionally, support tickets decreased across every category we measured.
137
+
138
+ What makes this approach different is that it represents a fundamental shift in how teams think about knowledge. Documentation is no longer an afterthought, it is a core deliverable. Experts agree that this reframing is essential for organizations navigating the complexity of modern software delivery.
139
+
140
+ In conclusion, good documentation pays for itself many times over.`;
141
+
142
+ const HUMAN = `The pump failed on a Tuesday. We pulled the housing and found the seal had cracked along one edge, most likely from the freeze in January that nobody thought to check for afterward.
143
+
144
+ Replacing it took an hour.
145
+
146
+ It has run clean since, though I want to check the pressure again after a month of real use, because a seal that fails once in a specific place tends to fail there again. That is the part the manual does not tell you. You learn it by having the same thing break twice and finally looking at where.
147
+
148
+ There is a second seal on the intake side that I have not touched. It looks fine. But the manual says both were replaced at the same time by whoever owned the place before us, which means both have the same hours on them, and one has already gone. So I ordered two.
149
+
150
+ We will know by March.`;
151
+
152
+ const esc = (s) => s.replace(/[&<>]/g, (c) => ({ "&": "&amp;", "<": "&lt;", ">": "&gt;" }[c]));
153
+ const spec = await fetch("./gates.json").then((r) => r.json());
154
+
155
+ const m = spec.meta || {};
156
+ document.getElementById("calib").innerHTML =
157
+ `Every cadence threshold was set by measuring that statistic across <b style="color:var(--ink)">${(m.n_documents || 0).toLocaleString()} human-written documents</b> ` +
158
+ `in three registers, then placed at the percentile that yields a <b style="color:var(--ink)">${Math.round((m.target_fpr || 0.05) * 100)}% false-positive rate</b>. ` +
159
+ `That rate is a property of the tool, not a hope.`;
160
+
161
+ const $ = (id) => document.getElementById(id);
162
+
163
+ function render(result) {
164
+ const out = $("out");
165
+ if (!result.findings.length) {
166
+ out.innerHTML = `<div class="clean"><b>Clean.</b> No gate fired. That is not a claim the writing is good, only that it does not contain the constructions this standard rejects.</div>`;
167
+ } else {
168
+ const verdict = result.errors ? "FAILS" : "passes with warnings";
169
+ const colour = result.errors ? "var(--error)" : "var(--warn)";
170
+ let html = `<div class="verdict" style="color:${colour}">${verdict} <small>${result.errors} error(s), ${result.warnings} warning(s) across ${result.words} words</small></div>`;
171
+ for (const f of result.findings) {
172
+ const where = f.line ? `line ${f.line}` : "whole document";
173
+ const body = f.excerpt
174
+ ? `<div class="excerpt">&ldquo;${esc(f.excerpt)}&rdquo;</div>`
175
+ : `<div class="excerpt" style="font-family:inherit">${esc(f.message)}</div>`;
176
+ html += `<div class="finding ${f.severity}">
177
+ <div class="tag">${f.severity} &middot; ${esc(f.gate)} &middot; ${where}</div>
178
+ ${body}<div class="fix">&rarr; ${esc(f.suggestion)}</div></div>`;
179
+ }
180
+ out.innerHTML = html;
181
+ }
182
+
183
+ let rows = "";
184
+ for (const c of result.cadence) {
185
+ if (c.threshold === null) {
186
+ rows += `<tr><td>${c.name}</td><td colspan="3" class="na">disabled &mdash; could not be calibrated</td></tr>`;
187
+ } else if (!c.measurable) {
188
+ rows += `<tr><td>${c.name}</td><td colspan="3" class="na">not measurable on this text</td></tr>`;
189
+ } else {
190
+ rows += `<tr><td>${c.name}</td><td class="num">${c.value.toFixed(3)}</td>
191
+ <td class="num na">${c.direction === "low" ? "&lt;" : "&gt;"} ${c.threshold.toFixed(3)}</td>
192
+ <td class="${c.fires ? "fires" : "passes"}">${c.fires ? "fires" : "ok"}</td></tr>`;
193
+ }
194
+ }
195
+ $("cadence").innerHTML = `<table><thead><tr><th>gate</th><th style="text-align:right">measured</th><th style="text-align:right">threshold</th><th></th></tr></thead><tbody>${rows}</tbody></table>`;
196
+ $("counts").textContent = `${result.words} words · ${result.sentences} sentences · ${result.paragraphs} paragraphs`;
197
+ }
198
+
199
+ function run() {
200
+ const text = $("input").value;
201
+ if (!text.trim()) {
202
+ $("out").innerHTML = `<p style="color:var(--muted);font-size:14px;">Paste some prose above.</p>`;
203
+ $("counts").textContent = "";
204
+ return;
205
+ }
206
+ render(check(text, spec));
207
+ }
208
+
209
+ $("run").onclick = run;
210
+ $("ex-machine").onclick = () => { $("input").value = MACHINE; $("cad-wrap").open = true; run(); };
211
+ $("ex-human").onclick = () => { $("input").value = HUMAN; $("cad-wrap").open = true; run(); };
212
+ $("clear").onclick = () => { $("input").value = ""; run(); };
213
+ $("input").addEventListener("keydown", (e) => {
214
+ if ((e.metaKey || e.ctrlKey) && e.key === "Enter") run();
215
+ });
216
+ </script>
217
+ </body>
218
  </html>