results: hardened set vs Grok Expert -- clean 8/8 contradictions, control arm confounded
LAWLESS1987+1: Claude Opus 4.8
97e4ab4
Confirmatory on the contradiction arm: 8/8 flagged with record-based reasoning
and no web search, including the LOW-overlap cases (X7 metric-conflation, X8
two-clause) that the 75% string-matcher and the 3/8 weak model miss -- evidence
of meaning-based detection. N1 (hardest high-overlap true near-miss) correctly
accepted.
Exploratory on the control arm -- two confounds spoil the false-alarm count:
(1) the 'accept into the record as written?' framing invites rejection of true
but incomplete statements (N4, U1) -- not contradiction flags;
(2) Grok Expert web-searched on some items and browsed this probe's own public
repo (github.com/LAWLESS1987) -- no longer pure record-reasoning (N3, U2).
N2 was mis-authored (actually false: e1hs2 clears the 30% bar) -> VOID per
criterion 4; author and Tetsu pre-check both missed it, the respondent caught it.
Clean re-run specified: disable web search, reframe to 'does this CONTRADICT?',
replace N2. U3/U4 not run (stopped at the confound).
C…
prereg: lock the hardened 16-case set before any scored run
LAWLESS1987+1: Claude Opus 4.8
938938a
Commits the sealed-packet hash (30c45d8a...) and all 16 statement hashes, the
event definitions, the pre-specified bar (>=6/8 hits AND <=1/8 false alarms AND
beat the 75% string-matcher baseline), roles, and VOID criteria -- before any
item is shown to a respondent, so the run is confirmatory not exploratory.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A standalone, version-controlled home for the contradiction-detection probe:
preregister -> seal (hashes only) -> present blind -> blind-score -> reveal and
re-hash. The author never scores; the scorer never sees types; the packet is
re-hashed at the reveal to prove nothing moved.
Honest-by-construction: every result is reported next to two baselines it must
beat -- a dumb string-matcher (capped at 75% on the hardened set, chance 50%) and
a weak 3B model (misses real contradictions rather than over-flagging). The
no-context / autonomous-detection arm is reported UNTESTABLE, not passed, because
no respondent yet holds the record unprimed.
Contents:
- seal.py / score.py / lexical_baseline.py / present.py (stdlib only)
- tests/test_score.py: 16/16, both-ways (a mutant that mis-credits UNRESOLVED
turns 4 checks red -- verified, then reverted)
- docs/METHOD.md: protocol + the role split + the 2026-10-22 pilot summary
- sealed/ is gitignored: case plaintexts, types, recordings, code maps…
