docs/DEVLOG.md

DEVLOG — build notes, decisions & challenges

A running record of how the triage copilot was built, the decisions made, and the challenges hit along the way. Written to feed the application's video walkthrough and writeup (the JD grades "clear communication of reasoning" and "trade-offs considered").

---

Working method

Everything is backed by an offline-first smoke harness (scripts/smoke_test.py) that runs in four layers and skips what it can't reach (heavy deps / no API key) rather than failing:

1. data layer (pydantic only) · 2. index + retrieval (chromadb) · 3. agent wiring (Pydantic AI TestModel, no API call) · 4. live triage (needs a key).

This let almost everything be verified for free — no API spend, no Ollama required for the deterministic checks. Live calls are reserved for explicit confirmation.

---

Phase 2 — agent + tools (summary + challenges)

Built schemas.py (typed TriageDecision), campaigns.py (loads a campaign and strips every _-prefixed key so private _design_note/_expected never reach the model), policy.py (parses policy.md into 26 citable rules), tools.py (RAG + mock sanctions + a deterministic risk scanner), and agent.py (Pydantic AI agent → structured TriageDecision).

Challenges hit and resolved:

• RunContext forward-ref crash. Tools were annotated RunContext[Deps], but RunContext was imported inside build_agent(). Pydantic AI resolves a tool's type hints against the module globals at registration time, so the annotation failed with NameError: RunContext. Fix: import Agent/RunContext at module top.
• ChromaDB client-lifecycle flakiness (the nastiest). store._client() created a new PersistentClient on every call, and the agent makes many tool calls per triage. Re-instantiating Chroma's client against the same path corrupts its shared system client, surfacing as intermittent Could not connect to tenant default_tenant / 'RustBindingsAPI' object has no attribute 'bindings' errors — passed once, failed the next run. Fix: cache the client + collections as per-process singletons (lru_cache); added a 12× repeat guard to the smoke test so it can't regress.
• Windows console encoding (cp1252). A summary print used →, which isn't in cp1252 → UnicodeEncodeError (em-dashes happen to be in cp1252, so they slipped by; the arrow didn't). Fix: ASCII in CLI prints + a UTF-8 stdout.reconfigure. Also silenced HF/transformers progress bars that were corrupting piped output and tripping grep's binary detection.
• Index citation duplication. The policy parser already keeps the rule ID inside rule.text, but build_index prepended it again (ELIG-2 — ELIG-2 — …). Fix: index rule.text directly.

Phase 2.5 — free local LLM toggle

Added a provider seam so triage can run on a free local Ollama model for dev, Claude for the demo. agent._resolve_model() reads CONFIG.llm_provider (anthropic default; --provider ollama per-run).

Challenges: pydantic-ai's OllamaProvider imports openai (not pulled by the [anthropic] extra) → added it. And OllamaProvider does not auto-append /v1 to the base URL — it passes it straight to the OpenAI client, so the code appends /v1 to OLLAMA_HOST. Verified end-to-end on qwen2.5:7b-instruct: camp-005 → REJECT (cited PROH-3), camp-017 → APPROVE. Quality is below Claude (occasionally an empty rationale), so local is the dev loop and the demo stays Claude.

---

Phase 3 — moderator review queue (this phase)

Goal: make the human/AI boundary visible. The AI recommends; a human decides and can override; every decision is logged. No auto-decide path.

Steps taken

1. src/audit.py — append-only JSON-Lines decision log: append_decision() (auto-stamps a UTC timestamp), load_decisions() (tolerates a missing/half-written file), decided_campaign_ids() (latest decision per campaign, to mark the queue). Flat file chosen over a DB: right-sized for the demo, trivially greppable, easy to show on video, clean seam to swap later.
2. src/policy.py — added policy_index() / get_rule(rule_id) (reusing parse_policy_rules) so the UI can expand a cited rule to its full policy text.
3. app.py — full rewrite from the placeholder into the queue: sidebar (provider toggle + index counts + audit history), a campaign selector marking decided items, a left/right split (campaign | AI recommendation), the decision card, and the human decision controls.
4. scripts/smoke_test.py — added deterministic checks for the audit round-trip and the rule lookup. Now 10 pass / 1 skip (live triage) with no key.
5. Verified: app parses, imports, both render functions execute (incl. nested columns + expanders), and a real headless streamlit run serves (/_stcore/health → ok).

Key decisions (and why)

• "What the AI couldn't verify" without a schema change. The card wants a "needs verification" section, but TriageDecision has no such field. Rather than change the schema (which ripples into the agent prompt + eval + local-model reliability), it maps to the existing questions_for_submitter, falling back to a low-confidence note. Reuse over new surface area.
• Triage cached in st.session_state, not st.cache_data. Streamlit reruns the whole script on every interaction; without caching, clicking any button would re-invoke (and re-bill) the agent. Storing the TriageDecision per campaign id in session state means one triage per campaign until an explicit "Re-run". Chose session state over @st.cache_data to avoid hashing a pydantic model and a (possibly network-bound) model object.
• Override gating is the governance story. The human decision maps to a recommendation (Approve↔APPROVE, Reject↔REJECT, Request-info↔ESCALATE). If it contradicts the AI, a written reason is required before the decision logs — the audit record carries is_override + reason. This is the most direct expression of "where AI should not replace human judgment."
• Provider toggle in the sidebar. Lets the whole queue be exercised for free on Ollama during dev, then flipped to Claude live for the recording, without editing .env.

Challenges hit

• Streamlit rerun model. Buttons trigger a top-to-bottom rerun, so widget read order matters: the reason text_area is defined before the decision buttons so its current value is available when a button handler runs in the same pass. Widgets are keyed per campaign id (reason_{cid}, ap_{cid}, …) so switching campaigns doesn't bleed state.
• Nested columns. The decision controls put st.columns(3) inside the right-hand column (one level of nesting). Older Streamlit forbade this; 1.40 allows a single level. Verified by executing the render functions directly in bare mode (no exception) before trusting it in the live app.
• Verifying a UI without clicking. Button clicks can't be automated here. Worked around it by (a) bare-importing app.py (runs the whole top-level script), (b) calling render_decision / render_human_controls directly with a sample TriageDecision to exercise the dynamic paths, (c) unit-testing the audit round-trip (incl. an override record), and (d) a real headless server boot. The only step left to a human is the final interactive click-through.
• Not spending the user's credits during tests. The full smoke test's live layer reads ANTHROPIC_API_KEY from .env; ran it with the key blanked (ANTHROPIC_API_KEY=) so the live check skips while everything else (10 checks) runs.

Verification status

scripts/smoke_test.py → 10 pass, 1 skip (live triage). App boots headless and serves. End-to-end data flow (triage → audit record incl. override) covered by the sum of the live-Ollama triage runs and the audit round-trip test. Remaining: human interactive click-through, then Phase 4 (eval) + Phase 5 (deploy).

---

Phase 3.5 — deterministic policy gate (the "not-a-wrapper" layer)

Why this phase exists. A review of the pipeline exposed the real weakness: despite the RAG, the risk scanner, the sanctions matcher and the typed schemas, nothing in code constrained the final decision — triage() returned the model's result.output verbatim. Every deterministic signal was fed to the model as advice; the policy invariants ("sanctions → escalate", "REJECT needs a cited hard rule", "low confidence on money → escalate", "injection → escalate") lived only as prose in the system prompt. So the adjudication was 100% the model's free judgment — which is both why it felt like a wrapper and why a weak local model (Ollama) and Claude diverged so much: swap the brain, swap the behavior, because the brain was the whole system.

The fix — src/gate.py. A deterministic gate sits between the model's TriageDecision and the final output. It recomputes the deterministic facts itself (it does not trust the model's self-report) — scan_risk_signals, check_sanctions, valid_rule_ids — and reconciles them against the model's recommendation. The envelope has exactly one rule: the gate may only route toward the human (→ ESCALATE). It never produces a more confident decision than the model (never APPROVE→REJECT, never ESCALATE→APPROVE, never invents a REJECT) and never overrides the human. The AI component can only ever defer harder to a person, never seize more authority.

Enforced invariants (each override cites the rule it enforces): sanctions match → ESCALATE (COMP-1); prompt injection → ESCALATE + the manipulation flag is corrected even if the model missed it, because the deterministic scanner is authoritative (DEC-6); a high-severity risk signal blocks an APPROVE (DEC-3, or COMP-2 for a high-value goal); low-confidence APPROVE → ESCALATE (DEC-5); and a REJECT without a valid hard citation → ESCALATE (DEC-2).

Calibration that matters. Only high-severity signals block an APPROVE. That keeps the showcase intact: camp-017 (debt principal) fires only interest_mentioned/unverified_organizer (both low) → APPROVE survives, while camp-005 (riba investment) fires investment_return (high) → an erroneous APPROVE would be caught. A REJECT resting on a real hard PROH-3 citation stands.

Payoff. The safety-critical behavior no longer depends on model IQ — Ollama and Claude converge on the cases that matter — and the gate's invariants are pure-Python, zero-API, so they double as the deterministic seed of the Phase 4 eval. The UI shows a gate banner whenever it adjusts a recommendation, and the audit log now records ai_llm_recommendation (raw) + gate_overrides alongside the final decision.

Challenges hit

• Model nondeterminism on the live showcase. Haiku sometimes returns a REJECT for camp-005 without a properly-typed hard citation; the gate then (correctly) routes it to ESCALATE per DEC-2. The live smoke assertion was relaxed from == "REJECT" to the model-independent invariant "never APPROVE" (REJECT or ESCALATE), since the deterministic gate test already proves the transition logic exactly.
• Schema confusion crashed whole runs. Haiku intermittently emitted severity="hard" on a RiskSignal (confusing it with RuleViolation's scale), exhausting Pydantic AI's retries and aborting the triage. Added a lenient field_validator on RiskSignal.severity that maps the common confusions (hard→high, soft→medium, …) — same "don't depend on the model being perfect" philosophy as the gate. (Separately observed, still open: Haiku occasionally passes phantom args to the zero-arg check_sanctions tool, and the documented intermittent Chroma RustBindings flake on Windows — both pre-existing, neither caused by the gate.)

Verification status

scripts/smoke_test.py → 12 pass under the venv (incl. a new 7-assertion deterministic gate test, the TestModel wiring through the gate, and live camp-005). Live CLI confirmed: camp-005 → REJECT preserved (gate agrees, no override); camp-009 → ESCALATE.

---

Phase 3.6 — React + FastAPI moderator console (replacing Streamlit as the shipped UI)

Why. Streamlit looks like a script UI; the demo video wanted something that reads as a real ops console. The triage/policy/audit code was already UI-agnostic (triage() returns a typed GatedDecision; audit/campaigns/policy are pure functions), so the move cost no changes to the AI layer — only a thin HTTP wrapper + a frontend.

Backend — api.py (FastAPI). Wraps src/ as /api/*. Pydantic models serialize for free. Two pieces of logic live here rather than in the client, on purpose:

• Override governance is server-enforced. POST /api/decisions looks up the cached triage, computes is_override, and returns 400 if a contradiction has no written reason. A UI cannot bypass the human/AI boundary — a stronger story than UI-only validation.
• Per-campaign triage cache (last-run-wins) so reruns/interactions don't re-bill; force=true re-runs. The endpoint also enriches each cited rule_violation/override with its policy rule text so the frontend needs no extra round-trips. In production FastAPI also serves the built SPA (StaticFiles mounted at / after the API routes), so the whole app is one origin in one container.

Frontend — frontend/ (Vite + React + TS + Tailwind). Deliberately lean for low-risk/fast: no router, no state library — useState/useEffect + a small typed fetch wrapper (api.ts), and hand-written TS mirrors of the schema (types.ts) instead of an OpenAPI codegen step. Single-screen console: queue → campaign detail → triage → human controls + audit drawer. The gate banner is the visual headline (indigo, "Policy gate adjusted {llm} → {final}" + cited rules). Tailwind class maps use full literal strings (lib.ts) so the content scanner keeps them.

Deploy — Docker on HF Spaces. Multi-stage Dockerfile (node builds the SPA → python serves). The Chroma index is rebuilt at image-build time (python -m scripts.build_index, local embeddings, no key) rather than committed, so there's no binary blob in git and no ingestion step on the Space. data/ made world-writable for the non-root HF user (audit log + Chroma sqlite). README metadata flipped to sdk: docker, port 7860.

Challenges / decisions

• npm --prefix on Windows read the repo-root package.json (which doesn't exist) instead of frontend/ — switched to running from the frontend/ dir.
• Build script is vite build (no tsc) so a stray type nit can't block the production build; types still guide the editor.
• Kept app.py. The Streamlit queue stays as a zero-dependency local fallback; the shipped UI is React, but if Node/Docker is unavailable the demo still runs.

Verification status

python -m scripts.test_api → 7/7 offline (incl. override-governance 400 + rule-text enrichment). npm run build clean (≈165 kB JS gzip 52 kB). A real uvicorn server serves /→200 (html), /assets/*→200, /api/*→200 from one origin; live /api/triage camp-017 → APPROVE (gate agrees). Remaining: docker build (daemon was off locally) + the in-browser click-through.

Phase 3.7 — policy reference drawer + clickable citations

Why. The decision card cites bare rule IDs (COMP-1, PROH-3). The five prefixes (ELIG / PROH / COMP / CONT / DEC) are internal jargon, and the only way to read a rule was to triage a campaign and expand that one citation — no way to learn the taxonomy or browse the policy. A moderator shouldn't have to guess what a code means.

What. A Policy button in the header opens a side drawer (mirrors the audit-history drawer) listing all 26 rules grouped by section, each section led by a plain-English name + one-line blurb. Every cited rule ID — in both the rule list and the gate banner — is now a clickable chip (RuleIdChip) that opens the drawer scrolled to and highlighting that exact rule (focusRule + scrollIntoView + a teal ring).

How / decisions.

• Backend stayed thin: src/policy.py gained SECTION_META (the prefix → name/blurb map — the headings in policy.md only carry names, so the blurbs live in code as the single source) and policy_sections(), which groups parse_policy_rules() by rule_id.split("-")[0]. One new route GET /api/policy. No AI/triage/gate code touched; zero new LLM spend.
• RuleRow was a single <button>; making the rule ID separately clickable would have nested a button in a button, so the row was split into a chevron toggle + the RuleIdChip + an evidence button. Same expand behavior, valid markup.
• Reused the AuditHistory aside w-80 border-l pattern wholesale; the two drawers coexist.

Verification. python -m scripts.test_api → 8/8 (new /api/policy: 5 sections in canonical order, every section named + described, rule count == valid_rule_ids()). npm run build clean (typecheck passes). scripts.smoke_test → 12/12 (one transient live-Haiku output-retry flake on a first run, green on re-run — orthogonal to this change). Remaining: in-browser click-through.

---

Phase 4 — evaluation harness + CI

Why. The JD asks for measurable quality that runs automatically. We had the seed of it (the policy gate's invariants were already pure-Python and unit-tested) but no harness that scored the agent against ground truth, and nothing in CI. This phase turns the 18 labelled campaigns into a test set and makes the safety properties a build gate.

Three layers (eval/run_eval.py, a full rewrite — the old file was the retired YouTube-RAG eval that imported src.rag):

1. Deterministic — the model-free safety gate (free, blocks CI). No LLM, no index, no key. Checks: (a) the privacy boundary — load_campaign lets no _-prefixed key reach the agent for any campaign; (b) ground-truth citation validity — every rule ID in every _expected block is a real ID in policy.md; (c) the policy-gate envelope — feed synthetic TriageDecisions through the real apply_policy_gate and assert the invariants from both directions: a sanctions hit (camp-009) can't be approved away (COMP-1), injection (camp-015) escalates and the manipulation flag is corrected even when the model set it False (DEC-6), low-confidence approve → human (DEC-5), a reject with no hard citation → escalate (DEC-2); and the envelope never over-fires — a clean approve survives, a well-founded PROH-2 reject passes through untouched, and an ESCALATE is terminal. Ten checks; exit non-zero on any failure.
2. Triage scoring (needs a key). Runs the real agent (triage(), gate included) over the test set and computes recommendation accuracy, escalation recall (overall + a safety-critical subset — the ESCALATE cases citing COMP-*/DEC-6/ELIG-4 — which is the metric the video quotes and CI --strict gates at 100%), reject precision + false-reject count (the dangerous error is rejecting a legitimate campaign, so false rejects are surfaced by id), and citation validity of the model's own citations. Each case is wrapped so one flaky run (the known Windows Chroma flake) records an ERROR row instead of losing the whole eval.
3. LLM-as-judge (--judge). Scores rationale faithfulness (is every claim supported by the cited rule text + campaign facts?) and calibration (does it escalate under genuine uncertainty rather than guess?) 1–5 on a subset, via the legacy src/llm provider Protocol.

Test set. eval/build_testset.py collects each campaign's _expected into a committed eval/testset.json (18 cases); --check mode fails CI if the file drifts from the campaigns, so the ground truth can't silently rot. The legacy testset.example.json (YouTube format) was removed.

CI — .github/workflows/eval.yml. On every push/PR: install deps, assert the test set is current, run the deterministic gate (blocking, free). A second step runs build_index + the LLM-judge on a 5-case subset only if an ANTHROPIC_API_KEY secret is present (if: env.… != ''), so forks and an unconfigured repo still go green, and the live model never gates the build or surprises us with spend. Results upload as an artifact.

Key decisions (and why)

• "Deterministic" = objectively-scored, not necessarily model-free — but the CI gate is model-free. The triage metrics are deterministic scoring of model output; producing that output needs the LLM. So CI's blocking gate is the layer that needs no model (boundary + citation validity + the gate envelope). That keeps the gate free, fast, and stable — it can't flake on model nondeterminism — which is exactly the property a safety gate should have.
• Probe the gate from both sides. It's not enough to prove the gate escalates the dangerous cases; an over-eager gate that escalated everything would also pass that. The envelope checks include "clean approve survives" and "founded reject survives" so the test pins the gate to only routing toward the human when a rule actually fires — the same property that keeps the camp-017 showcase APPROVE alive.
• Safety-critical recall is the gate metric, not raw accuracy. A weak model will get the category of a clean approve wrong sometimes; that's tolerable. Missing a sanctions/injection escalation is not. --strict fails only on the safety subset, invalid citations, or a false reject — never on a benign accuracy miss.

Verification status

eval.run_eval --deterministic-only → 10/10 PASS, exit 0, offline (no key, no index, no pydantic_ai import — the agent import is lazy so the free gate stays light). build_testset --check green. Full triage layer exercised end-to-end on the free local model (--provider ollama, zero API spend) to confirm the scoring/metrics/judge code paths and the gate's robustness on a weak brain. scripts.smoke_test still 12/12. Remaining: a full Anthropic run for the demo's headline numbers, and adding the ANTHROPIC_API_KEY secret in GitHub so the CI judge step activates.

---

Phase 4.1 — Gate hardening (2026-06-06)

The Phase-4 eval did its job: a weak-model (Ollama) run, plus the user's live console click-through, surfaced two genuine holes in the policy gate. Closed both in src/gate.py without widening the gate's envelope (it still only ever routes toward the human).

Finding #1 — ELIG-4 not enforced on APPROVE

The gate blocked an APPROVE only on high-severity risk signals and on low confidence. But the fund-use-breakdown signal large_goal_no_breakdown_check is medium severity, so a >$10k APPROVE with no breakdown (camp-011/012) slipped straight through. Fix: a _BREAKDOWN_SIGNAL trigger in the APPROVE block → ESCALATE citing ELIG-4. The scanner can't confirm a breakdown is actually present (the signal fires on goal amount alone), so every large-goal approve defers to a human to check one — calibrated humility, not rejection. The >=$50k band is already covered by high_value_goal→COMP-2, so the two bands are contiguous.

Finding #2 — gate trusted a hard citation's existence, not its correctness

A weak model fabricated a hard category to force a REJECT (camp-018 invented COMP-3; camp-011 in the user's live run invented PROH-2/PROH-4 on a vague-beneficiary campaign), and the old _has_valid_hard_citation preserved the REJECT because the cited ID was a real hard rule. Decided policy + fix: _has_confirmed_hard_citation requires a hard citation to be corroborated by the deterministic scanner via _HARD_RULE_SIGNALS (PROH-2→weapons, PROH-3→investment_return, PROH-4→prize_draw, PROH-7→investment_return, COMP-3→off_platform_payment). All four genuine-reject campaigns carry their corroborating signal, so real rejects survive; an uncorroborated hard citation escalates (DEC-2). Content rules with no deterministic detector (PROH-1 illegal, PROH-5 hate, PROH-6 adult) are not confirmable, so a citation alone no longer holds the REJECT — it escalates rather than trust the model's word.

Why this is still inside the envelope. The gate does not re-adjudicate whether the cited rule substantively applies (that would be the gate overruling the model's reasoning). It only demands independent deterministic evidence before letting the AI's REJECT stand un-escalated. Where the evidence is absent, it does the one thing the gate is always allowed to do: hand the case to a human.

Tests

• scripts/smoke_test.py::t_policy_gate +2 assertions (uncorroborated PROH-2 reject → ESCALATE; large-goal APPROVE → ELIG-4).
• eval/run_eval.py deterministic layer 10 → 12 checks: gate_founded_reject_survives moved onto camp-008 (corroborated PROH-2); added gate_uncorroborated_reject_escalates (camp-011) and gate_large_goal_approve_escalates (camp-012); gate_clean_approve_survives moved off camp-001 ($14k — now correctly ELIG-4-blocked) onto the camp-017 showcase ($5.5k).

Verification status

eval.run_eval --deterministic-only → 12/12 PASS offline. scripts.smoke_test → 12/12 under the venv, incl. live camp-005 Claude triage exercising the new corroboration path. scripts.test_api → 8/8. build_testset --check clean. camp-017 debt-principal showcase APPROVE preserved. Remaining: commit + push (CI re-run), then a full --judge Anthropic eval for the demo's headline numbers.