docs: cold-start measurement + falsified-assumption finding + v1.1 contingency

HF Spaces cold-start measured at N=3 samples on 2026-04-15: median
113 s, range 89-129 s. Both tails fire the 60 s pre-committed
contingency gate from the preceding "Cold-start contingency" entry.

Key finding: the pre-committed fix (lazy-load K8s corpus) was written
under an assumption about cost attribution — corpus loading is the
dominant cold-start cost. Measurement falsified this. K8s corpus load
contributes <1 s of the 113 s median; the dominant cost is ~90 s of
silent Python init (interpreter start + module imports + initial
model weight loading). Executing the original fix would save ~1 % of
the overshoot.

Action taken: document the falsified assumption, accept the measured
baseline, re-pre-commit a new contingency targeting the actual
dominant cost (lazy-load cross-encoder + embedder + injection
classifier to first-relevant-request, traffic-justified trigger) so
the decision is not relitigated at review time. The contingency
trigger is deliberately left unnamed in advance — the entire point
of this entry is that estimate-justified triggers invite the same
falsification pattern.

Methodology lesson: honoring a pre-committed gate means honoring its
intent (prevent motivated reasoning about pass/fail), not mechanically
executing the pre-committed recipe regardless of whether its
empirical premise has survived measurement.

Raw measurement logs preserved under measurements/ so the specific
quantitative claims in the DECISIONS.md entry can be cross-checked
against the underlying evidence without re-running.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

DECISIONS.md

@@ -608,6 +608,127 @@ numbers needed to make the decision; the decision is documented here
 so future-me remembers the threshold and doesn't optimize prematurely
 on a hunch.
 
+## Cold-start gate fired — assumption falsified, fix deferred to v1.1 at the right cause
+
+The preceding "Cold-start contingency" entry pre-committed a lazy-load
+fix (FastAPI eager, K8s lazy on first request) if the measured cold
+start exceeded 60 seconds. Measurement falsified the entry's core
+assumption: **corpus loading is not the dominant cold-start cost**.
+The committed fix addresses ~1 % of the observed overshoot. Executing
+it verbatim would honor the gate's letter but not its intent — theater
+dressed as discipline. This entry documents the measurement, the
+falsified assumption, and the new contingency pre-committed at the
+actual cause.
+
+**Measurement (N=3, 2026-04-15, HF Spaces target deployment):**
+
+| Sample | Cold start | Silent Python init | Visible phase |
+|---|---|---|---|
+| N=1 | 113 s | ~101 s | ~12 s |
+| N=2 |  89 s |  ~70 s | ~19 s |
+| N=3 | 129 s | ~115 s | ~14 s |
+
+- Median 113 s, mean ~110 s, range 89–129 s (spread ~40 s)
+- **Gate fire is unambiguous at both tails.** Even the fastest sample
+  (89 s) is ~48 % over the 60 s threshold; the slowest (129 s) is
+  ~115 % over. No boundary ambiguity.
+- **Sample-size justification.** N=3 is acknowledged as a small sample.
+  It is adequate here because (a) the gate-fire conclusion is stable
+  across both tails, (b) the "silent Python init dominates variance"
+  finding is stable across all three samples (silent phase varies
+  70 → 115 s across runs; visible phase varies only 12 → 19 s), and
+  (c) the cost of additional samples (manual HF Space restart + ~2 min
+  wait + log extraction per sample) exceeds the marginal information
+  gain once both tails fire the gate and the variance pattern is stable.
+  N=4 would tighten the confidence interval on the median but does not
+  change either the gate-fire conclusion or the falsified-assumption
+  finding.
+- **Variance source named.** HF Spaces shared-infrastructure CPU / IO
+  contention during Python module imports. The silent-init phase
+  varies 45 s across samples (70 → 115 s); the visible phase is stable
+  (12–19 s). That is the signature of host-level contention on a
+  shared physical node, not code-level variability. An
+  exclusively-owned container would plausibly show a tighter bound.
+- **Raw log captures** (preserved so this entry can be cross-checked
+  against the underlying evidence without re-running the measurement):
+  `measurements/2026-04-15-coldstart-n1.log`, `-n2.log`, `-n3.log`.
+
+**Where the cost lives.** At the median (113 s):
+
+- **Silent Python init phase — ~90 s (≈ 80 % of total):** interpreter
+  start, module imports (`torch`, `transformers`, `langchain`, `faiss`,
+  `fastapi`, `httpx`, the full dependency closure), and initial model
+  weight loading (`all-MiniLM-L6-v2` embedder, cross-encoder
+  reranker). Not logged — no observability inside the import chain.
+- **Visible startup phase — ~15 s (≈ 15 % of total):** injection
+  classifier init (~10 s, includes the "classifier skipped" warning),
+  FastAPI corpus load (< 1 s, +0.9 MB RSS), K8s corpus load (< 1 s,
+  +25.8 MB RSS), reranker warmup (~2 s).
+
+**The K8s corpus load — which the pre-committed fix was designed to
+defer — contributes under 1 second of the 113-second median.**
+Deferring it saves roughly 1 % of the overshoot. FastAPI corpus load
+is the same order of magnitude. Corpus loading is simply not where the
+cost lives on this deployment.
+
+**Why we are not executing the pre-committed fix.** The preceding
+contingency was written under an empirical assumption about cost
+attribution (corpus loading is the dominant cost). Measurement
+falsified the assumption. Implementing the fix anyway would be a
+mechanical execution of a recipe whose premise has been disproven —
+it checks the gate-honoring box while failing to address the cause.
+That is structurally identical to relaxing-by-redefinition ("60 s was
+too tight"), just in the opposite direction: **relaxing by execution**.
+The pre-commitment rule's purpose is to prevent motivated reasoning
+about the gate, not to mandate mechanical compliance with a recipe
+whose empirical foundation has collapsed.
+
+The honest action is (1) accept the measurement as the v1 baseline,
+(2) document the falsified assumption explicitly (this entry),
+(3) re-pre-commit a new contingency at the actual dominant cost with
+an explicit trigger condition so the decision is not relitigated at
+review time, and (4) update the user-facing README surface to reflect
+the measured cold-wake number rather than the optimistic pre-deploy
+estimate.
+
+**v1.1 contingency — pre-committed:**
+
+> **If HF Spaces traffic produces more than N cold wakes per day**
+> (N to be determined from observed usage patterns after launch, **not
+> estimated in advance**), defer eager loading of (a) the cross-encoder
+> reranker, (b) the sentence-transformers embedder, and (c) the
+> injection classifier tier to first-relevant-request.
+>
+> **Estimated work:** 4–6 hours (lazy-init wrappers + first-request
+> caching + integration tests for the warm/cold transition).
+>
+> **Expected tradeoff:** cold wake ~113 s → ~50–60 s (approaches the
+> original 60 s target); **first request after any cold wake incurs
+> +8–15 s** additional latency (model weights load synchronously in
+> the request path), after which subsequent warm requests return to
+> normal ~5 s latency.
+>
+> **Trigger is usage-justified, not estimate-justified.** Until real
+> traffic data justifies the work, there is nothing to optimize — a
+> recruiter demo that gets one cold wake per day does not pay for
+> 4–6 hours of engineering plus the new first-request-latency failure
+> mode. The trigger threshold N is left unnamed deliberately: naming a
+> number in advance would invite the same falsification pattern this
+> entry is documenting.
+
+**Methodology lesson.** When a pre-committed contingency is written
+under an empirical assumption, the contingency only holds if the
+assumption survives measurement. If measurement falsifies the
+assumption, the correct action is to document the falsification,
+accept the observed baseline, and re-pre-commit at the actual cause.
+The wrong action is to execute the original recipe anyway, which
+trades one form of motivated reasoning (threshold relaxation) for
+another (recipe compliance). The underlying discipline — "pre-commit
+your gates and honor them" — does not mean "mechanically run the
+pre-committed fix regardless of what it addresses." It means "honor
+the gate's *intent*, which is to prevent motivated reasoning about
+pass/fail."
+
 ## False-premise questions come in two flavors
 
 When authoring golden-dataset questions whose premise is wrong, the

measurements/2026-04-15-coldstart-n1.log

@@ -0,0 +1,26 @@
+# HF Spaces cold-start measurement N=1
+# Source: HF Spaces runtime log, container startup trace
+# Container: huggingface.co/spaces/Nomearod/agentbench
+# Build: first v1 deploy (commit 6955d72 on hf/main, main at 8974e47 + frontmatter injection)
+# Cold-start delta: 09:40:34 − 09:38:41 = 113 seconds
+
+===== Application Startup at 2026-04-15 09:38:41 =====
+
+2026-04-15 09:40:22 [warning  ] injection_classifier_no_url    msg="Tier 'classifier' configured but classifier_url is empty; classifier tier will be skipped at runtime."
+2026-04-15 09:40:22 [warning  ] injection_classifier_no_url    msg="Tier 'classifier' configured but classifier_url is empty; classifier tier will be skipped at runtime."
+2026-04-15 09:40:31 [warning  ] audit_hmac_key_missing         msg='No HMAC key provided; using random per-process key. IP hashes will not be stable across restarts or instances. Set AUDIT_HMAC_KEY env var or pass hmac_key for stable audit correlation.'
+2026-04-15 09:40:31 [info     ] corpus_loaded                  label='FastAPI Docs' name=fastapi providers=['openai', 'anthropic'] rss_delta_mb=0.9 rss_mb=810.8 store_path=.cache/store
+2026-04-15 09:40:31 [info     ] corpus_loaded                  label=Kubernetes name=k8s providers=['openai', 'anthropic'] rss_delta_mb=25.8 rss_mb=835.6 store_path=.cache/store_k8s
+2026-04-15 09:40:31 [info     ] multi_corpus_mode              corpora=['fastapi', 'k8s'] default=fastapi providers=['openai', 'anthropic']
+INFO:     Started server process [1]
+INFO:     Waiting for application startup.
+2026-04-15 09:40:31 [info     ] warmup_start
+2026-04-15 09:40:32 [info     ] reranker_loading               model=cross-encoder/ms-marco-MiniLM-L-6-v2
+2026-04-15 09:40:34 [info     ] warmup_complete
+INFO:     Application startup complete.
+INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
+
+# Phase breakdown:
+#   Silent Python init:    09:38:41 → 09:40:22 = 101s (interpreter start, module imports, initial model weights)
+#   Visible phase:          09:40:22 → 09:40:34 = 12s (injection classifier warnings + corpus loads + reranker warmup)
+#   Cold-start total:                              113s

measurements/2026-04-15-coldstart-n2.log

@@ -0,0 +1,26 @@
+# HF Spaces cold-start measurement N=2
+# Source: HF Spaces runtime log, container startup trace
+# Container: huggingface.co/spaces/Nomearod/agentbench
+# Build: post-audit-path fix rebuild (commit 55afe8a on hf/main)
+# Cold-start delta: 10:37:59 − 10:36:30 = 89 seconds
+
+===== Application Startup at 2026-04-15 10:36:30 =====
+
+2026-04-15 10:37:40 [warning  ] injection_classifier_no_url    msg="Tier 'classifier' configured but classifier_url is empty; classifier tier will be skipped at runtime."
+2026-04-15 10:37:40 [warning  ] injection_classifier_no_url    msg="Tier 'classifier' configured but classifier_url is empty; classifier tier will be skipped at runtime."
+2026-04-15 10:37:57 [warning  ] audit_hmac_key_missing         msg='No HMAC key provided; using random per-process key. IP hashes will not be stable across restarts or instances. Set AUDIT_HMAC_KEY env var or pass hmac_key for stable audit correlation.'
+2026-04-15 10:37:57 [info     ] corpus_loaded                  label='FastAPI Docs' name=fastapi providers=['openai', 'anthropic'] rss_delta_mb=0.9 rss_mb=811.5 store_path=.cache/store
+2026-04-15 10:37:57 [info     ] corpus_loaded                  label=Kubernetes name=k8s providers=['openai', 'anthropic'] rss_delta_mb=25.8 rss_mb=836.3 store_path=.cache/store_k8s
+2026-04-15 10:37:57 [info     ] multi_corpus_mode              corpora=['fastapi', 'k8s'] default=fastapi providers=['openai', 'anthropic']
+INFO:     Started server process [1]
+INFO:     Waiting for application startup.
+2026-04-15 10:37:57 [info     ] warmup_start
+2026-04-15 10:37:57 [info     ] reranker_loading               model=cross-encoder/ms-marco-MiniLM-L-6-v2
+2026-04-15 10:37:59 [info     ] warmup_complete
+INFO:     Application startup complete.
+INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
+
+# Phase breakdown:
+#   Silent Python init:    10:36:30 → 10:37:40 = 70s   (interpreter start, module imports, initial model weights)
+#   Visible phase:          10:37:40 → 10:37:59 = 19s  (injection classifier warnings + corpus loads + reranker warmup)
+#   Cold-start total:                              89s

measurements/2026-04-15-coldstart-n3.log

@@ -0,0 +1,26 @@
+# HF Spaces cold-start measurement N=3
+# Source: HF Spaces runtime log, container startup trace
+# Container: huggingface.co/spaces/Nomearod/agentbench
+# Build: same image as N=2 (commit 55afe8a on hf/main); manual restart via HF Space settings
+# Cold-start delta: 10:49:07 − 10:46:58 = 129 seconds
+
+===== Application Startup at 2026-04-15 10:46:58 =====
+
+2026-04-15 10:48:53 [warning  ] injection_classifier_no_url    msg="Tier 'classifier' configured but classifier_url is empty; classifier tier will be skipped at runtime."
+2026-04-15 10:48:53 [warning  ] injection_classifier_no_url    msg="Tier 'classifier' configured but classifier_url is empty; classifier tier will be skipped at runtime."
+2026-04-15 10:49:05 [warning  ] audit_hmac_key_missing         msg='No HMAC key provided; using random per-process key. IP hashes will not be stable across restarts or instances. Set AUDIT_HMAC_KEY env var or pass hmac_key for stable audit correlation.'
+2026-04-15 10:49:05 [info     ] corpus_loaded                  label='FastAPI Docs' name=fastapi providers=['openai', 'anthropic'] rss_delta_mb=1.0 rss_mb=811.7 store_path=.cache/store
+2026-04-15 10:49:05 [info     ] corpus_loaded                  label=Kubernetes name=k8s providers=['openai', 'anthropic'] rss_delta_mb=25.8 rss_mb=836.5 store_path=.cache/store_k8s
+2026-04-15 10:49:05 [info     ] multi_corpus_mode              corpora=['fastapi', 'k8s'] default=fastapi providers=['openai', 'anthropic']
+INFO:     Started server process [1]
+INFO:     Waiting for application startup.
+2026-04-15 10:49:05 [info     ] warmup_start
+2026-04-15 10:49:06 [info     ] reranker_loading               model=cross-encoder/ms-marco-MiniLM-L-6-v2
+2026-04-15 10:49:07 [info     ] warmup_complete
+INFO:     Application startup complete.
+INFO:     Uvicorn running on http://0.0.0.0:7860 (Press CTRL+C to quit)
+
+# Phase breakdown:
+#   Silent Python init:    10:46:58 → 10:48:53 = 115s  (interpreter start, module imports, initial model weights)
+#   Visible phase:          10:48:53 → 10:49:07 = 14s  (injection classifier warnings + corpus loads + reranker warmup)
+#   Cold-start total:                              129s

measurements/README.md

@@ -0,0 +1,14 @@
+# measurements/
+
+Raw measurement artifacts referenced from DECISIONS.md entries.
+
+Each file is the raw observation (log snippet, trace, or metric dump)
+that backs a specific quantitative claim in DECISIONS.md. Keeping the
+raw data here lets a future reader cross-check any DECISIONS.md number
+against its underlying evidence without having to re-run the
+measurement or trust the narrative summary.
+
+Naming: `YYYY-MM-DD-<topic>-<variant>.log`
+
+Current entries:
+- `2026-04-15-coldstart-n1.log`, `-n2.log`, `-n3.log` — HF Spaces cold-start samples N=1..3. Backs the DECISIONS.md entry "Cold-start gate fired — assumption falsified, fix deferred to v1.1 at the right cause."