worldmonitor/docs/archive/plans/unify-cii-single-source.md

Unify the CII — one engine, server-canonical

Origin: Issue #3738 ("Security dimension is 100% GPS jamming"). Investigation showed the mislabel is a symptom, not the bug. The bug: WorldMonitor runs two CII engines that diverge in formula, inputs, and output shape. This plan unifies them.

Reviewed 2026-05-22 by three parallel reviewers (feasibility / adversarial / coherence). Their verified findings are folded in. Corrections to the first draft are marked [rev].

Problem

	Frontend engine	Server engine
File	src/services/country-instability.ts	server/worldmonitor/intelligence/v1/get-risk-scores.ts
Scoring fns	calcUnrestScore, calcConflictScore, calcSecurityScore, calcInformationScore, calcNewsConflictFloor	4 inline component blocks in computeCIIScores
Inputs	~20 ingest*ForCII functions	~12 AuxiliarySources (post-Phase-1: ~17)
Output consumers	calculateCII() / getCountryScore()	RPC getRiskScores → MCP get_country_risk, API, panels
Cadence	recomputed live as browser streams tick	cached risk:scores:sebuf:v2, TTL 600 s

Three independent divergences, not one:

1. Input divergence. The server's computeCIIScores does not score 8 signal families the frontend scores (military flights, military vessels, aviation disruptions, AIS disruptions, earthquakes, sanctions, temporal anomalies, HAPI). [rev] "Does not score" ≠ "cannot see" — 6 of the 8 already have a usable server source (audit table ✅), AIS disruptions has a relay source needing only geo-attribution, and only military vessels need a genuinely new classifier. See the audit table.
2. Formula divergence. Beyond inputs, the blend formulas differ structurally. Server get-risk-scores.ts:549-556: `baseline·0.4 + eventScore·0.6 + climateBoost + cyberBoost
   • fireBoost + advisoryBoost + orefBlendBoost + displacementBoost. Frontend country-instability.ts:~1014: adds hotspotBoost, newsUrgencyBoost, focalBoost, supplementalSignalBoost, getEarthquakeBoost, getSanctionsBoost, and uses a *different* displacementBoostshape (server =log10 ramp cap 20; frontend = step function cap 8). The component sub-formulas (securityGPS-only vs 4-input) also differ. And the *frontend itself* carries two divergent blends:calculateCII(definedcountry-instability.ts:975, blend at :1014) includes the earthquake/sanctions/supplemental boosts; getCountryScore (:1042) omits all three. Phase 3a's canonical-decision list must pick one blend — Phase 4 then silently changes scores for getCountryScore` consumers (an expected, intended change).
3. Output-shape divergence. The CiiComponents proto fields are newsActivity / ciiContribution / geoConvergence / militaryActivity — positionally aliased to information / unrest / conflict / security by cached-risk-scores.ts:92-97 and get-risk-scores.ts:572-577. There is no security proto field.

Because both engines are user-facing (MCP/API serve the server score; the UI's calculateCII() / getCountryScore() serve the frontend score), the MCP tool and the on-screen CII can disagree for the same country.

Scope boundary — "unify" is not "correct"

This plan makes the two engines produce one score. It does not fix whether that score is right. Specifically: calcSecurityScore (flights + vessels + aviation + GPS) is military activity, not "security" in the terrorism sense (issue #3738's actual complaint). Unifying on it freezes that model. Correcting the security model is out of scope here, but must not be silently dropped: filing the follow-up issue — a real security/terrorism signal, with a named owner — is a Phase 0 deliverable, and #3738 is closed against that follow-up, not against this plan's rename. Phase 0's label rename ("Security" → "Mil. Activity") is the honest stopgap that makes the frozen model's name accurate; it is a deliberate, lasting decision, not a placeholder. [rev] This is why Phase 0 is safe to ship early: the plan commits to "the dimension stays military-activity-only," so the label will not need re-reverting later.

Target architecture

The server get-risk-scores.ts is the single CII engine. The browser renders its output.

• Delete calculateCII(), getCountryScore(), calc*Score, calcNewsConflictFloor, and the CII-scoring state from country-instability.ts.
• All frontend CII consumers read the server RiskScores proto via the already-built src/services/cached-risk-scores.ts adapter (getCachedScores, toCountryScore, fetchCachedRiskScores, circuit breaker, localStorage persistence). [rev] This module already exists and several call sites already dual-path through it — Phase 4 is "remove the frontend-engine arm," not "build a consumption layer."
• Accepted tradeoff: the CII number freshness becomes the server cadence (~10 min), not live. Map layers stay live; only the index number follows server cadence. For a baseline·0.4 + eventScore·0.6 slow index this is acceptable — but the acceptance is per-consumer, not blanket. Live-CII-dependent surfaces and their verdict:
  • checkCIIChanges instability alerts (cross-module-integration.ts:561) — delta detection moves from live ticks to 10-min snapshots; needs retuning (see Risk 4).
  • getCountryScore map tint (DeckGLMap.ts / Map.ts via setCIIGetter) — 10-min staleness acceptable for a country-level choropleth.
  • story-data.ts, military-surge.ts — consume CII for context, not real time; 10-min acceptable. Phase 4 confirms each consumer owner signs off; the alerts surface is the one with a real behavior change, not a pure latency change.

A shared-formula module ("one formula, two engines") is rejected — the user's decision is one engine. Note: a shared module would eliminate formula divergence but not input or country-set divergence, so it does not by itself deliver one CII.

Per-signal source audit

The 8 families the server CII does not currently score, and what each needs:

Signal	Frontend ingest	Server source today	Work
Aviation disruptions	ingestAviationForCII	aviation:delays-bootstrap:v2 (Redis — pre-merged FAA + intl + NOTAM by seed-aviation.mjs) ✅	Plumb into AuxiliarySources
AIS disruptions	ingestAisDisruptionsForCII	get-vessel-snapshot.ts emits AisDisruption (live relay HTTP, not a Redis key) ⚠️	Phase 2 — cached to a Redis key by Phase 2's scheduled relay job, then plumbed + geo→country
Earthquakes	ingestEarthquakesForCII	seismology:earthquakes:v1 (Redis) ✅	Plumb + geo→country
Sanctions	ingestSanctionsForCII	sanctions/v1/list-sanctions-pressure.ts (per-country) ✅	Plumb
Temporal anomalies	ingestTemporalAnomaliesForCII	server-detected (TemporalAnomalyProto, temporal-baseline.ts consumeServerAnomalies()) ✅	Plumb
Military flights	ingestMilitaryForCII	military:flights:v1 (Redis) — already operator-classified by scripts/seed-military-flights.mjs ✅	Per-country count + location/foreign-presence attribution
Military vessels	ingestMilitaryForCII	relay get-vessel-snapshot.ts has raw vessels; no military classification/attribution ⚠️	Port classifier (MILITARY_VESSEL_PATTERNS, KNOWN_NAVAL_VESSELS in src/config/military.ts) + attribution
HAPI conflict	ingestHapiForCII	conflict/v1/get-humanitarian-summary-batch.ts → conflict:humanitarian:v1:{ISO3} ✅	Phase 1 plumb — or take the drop decision in Phase 3a (frontend uses it only as ACLED-empty fallback)

[rev] Corrections from the first draft: only military vessels need genuinely new classification work — military flights are already classified in military:flights:v1; the relay is a process (scripts/ais-relay.cjs) reached by per-request HTTP, not a Redis key; AIS disruptions and earthquakes both need a geo→country step. No new persistent- WebSocket worker is needed (the relay process already holds the streams), but "plumbing" for AIS still means a live relay HTTP call with a timeout, not a cheap cache read.

Additional frontend-only inputs (feed calculateCII but not in the 8 families): focalPointDetector.getCountryUrgencyMap() (country-instability.ts:977), hotspot boost (getHotspotBoost), learning-mode gating (isInLearningMode). Decide per Phase 3a.

Country-set divergence [rev]

Server computeCIIScores iterates Object.keys(TIER1_COUNTRIES) — 31 countries (_shared.ts:35, includes KR/IQ/AF/LB/EG/JP/QA). Frontend calculateCII() iterates new Set([...countryDataMap.keys(), ...Object.keys(CURATED_COUNTRIES)]) — the same 31 curated plus any country that received ingested data (dynamic). So after Phase 4, non-curated countries with events lose their CII. This is a smaller gap than a reviewer first claimed (it is not 31→24), but it is real: decide in Phase 3a whether to expand the server set or accept that CII is curated-countries-only (those dynamic scores were thin — baseline 20 + events — anyway).

Phases

Phases 0, 1, 2, 3b, and 4 are each an independently shippable PR; Phase 3a is a design-note decision table, not a PR. The Guardrails section below is not a phase — each test ships inside an earlier phase's PR (noted per test).

Phase 0 — Label stopgap (ships immediately, independent)

• Rename the CII component label "Security" → "Mil. Activity" in all three component blocks of src/locales/en.json (lines 101, 745, 3170) and the 20 non-English locale files in src/locales/*.json (regenerate any .d.ts companions).
• Internal dimension key stays security; only the display string changes.
• File the security-model follow-up issue (#3738's actual complaint — a real security/terrorism signal) and link #3738 to it. Owner: TBD — assign before Phase 0 ships; Phase 0 is not complete with only the rename.
• No engine change. Safe given the Scope-boundary commitment above.

Phase 1 — Server acquires the cheap signals

• AuxiliarySources + fetchAuxiliarySources(): add aviation (aviation:delays-bootstrap:v2 — the pre-merged FAA + intl + NOTAM source), earthquakes (seismology:earthquakes:v1, with geo→country), sanctions (sanctions:pressure:v1), temporal anomalies (temporal:anomalies:v1). All four are backed by an existing Redis key.
• CountrySignals + emptySignals(): add the count fields (aviation{Closure,Severe,Major,Moderate}Count, earthquake{Significant,Major,Severe}Count, sanctions{Entry,NewEntry}Count, temporalAnomaly{,Critical}Count).
• Wire each into the per-country accumulation loop. No scoring change — signals gathered but unused. Additive and safe.
• AIS disruptions moved to Phase 2. Unlike the four above, AIS disruptions have no Redis key — get-vessel-snapshot.ts fetches them from the Railway relay per request. Adding a live relay HTTP call into the 600 s scoring path is the anti-pattern the feasibility review flagged; instead AIS rides on the scheduled relay job Phase 2 already stands up.

Phase 2 — Server military flights, vessels + AIS disruptions

• Per-country aggregation of military:flights:v1 (already operator-classified) — add location-code attribution for foreign presence.
• New server-side military vessel classifier: port MILITARY_VESSEL_PATTERNS / KNOWN_NAVAL_VESSELS and apply to the relay vessel snapshot. This is the single largest new-build item in the plan — before committing, Phase 3a must check the vessel signal's weight in the CII blend; if it is marginal, drop military vessels and document the accepted input gap is a valid Phase 3a decision (mirroring HAPI's "or drop").
• Port the intent of ingestMilitaryForCII's foreign-presence weighting — not the representation hack. The frontend fakes the ×2 weight by pushing synthetic {} objects so array.length inflates (country-instability.ts:447-456); the server must store honest counts and apply the ×2 in the formula, or any future reader of the new key gets a ~3× inflated aircraft count.
• Write per-country counts to a Redis key (e.g. intelligence:military-cii:v1).
• Decision: aggregation runs as a scheduled server job writing the Redis key (not a relay extension) — keeps get-risk-scores.ts reading Redis uniformly, matches the 600 s cadence, and avoids coupling scoring to relay request latency.
• AIS disruptions (moved from Phase 1). AIS disruptions have no Redis key — get-vessel-snapshot.ts fetches them from the relay per request. The same scheduled job caches them to a Redis key with geo→country attribution (AisDisruption carries {lat,lon} + a free-text region, no countryCode), so fetchAuxiliarySources() reads them uniformly like the Phase 1 signals — no live relay call in the scoring path. Add aisDisruption{High,Elevated,Low}Count to CountrySignals.
• Attribution caveat: the server's geoToCountry (get-risk-scores.ts:154) is a bounding-box scan; the frontend uses polygon containment. For straits/borders (Hormuz, Taiwan Strait, Black Sea — the high-signal cases) bbox will mis-attribute. Either port polygon containment server-side or accept and document the precision loss.

Phase 3a — Reconciliation decisions (design note, no code)

"Port verbatim" is not sufficient — the formulas diverge structurally and at least one divergence is intentional. Before the behavior-changing PR opens, settle every decision below and record it in a signed-off table. Phase 3b executes this table; it does not make decisions inside the PR.

• For each of the 4 component formulas and each blend boost (hotspotBoost, newsUrgencyBoost, focalBoost, supplementalSignalBoost, earthquakeBoost, sanctionsBoost, displacementBoost), a canonical decision: which engine's formula wins. Default = frontend, except where the server intentionally diverged — e.g. the information cap was deliberately raised 20→100 in issue #3739 (get-risk-scores.ts:518-521); the server version wins there.
• The BASELINE_RISK / EVENT_MULTIPLIER scalar-table reconciliation.
• The getScoreLevel threshold reconciliation — frontend getLevel (≥81 / ≥66 / ≥51 / ≥31) vs server adapter cached-risk-scores.ts getScoreLevel (≥70 / ≥55 / ≥40 / ≥25). Even identical scores render different level badges until this is unified.
• focal-urgency / hotspot-boost / learning-mode: port if reproducible from server-available signals, drop otherwise.
• The country-set question (Risk 6: expand server set, or accept curated-only).
• The Phase 3-proto field-naming choice (below).
• The Risk 3 cold-cache decision.
• Signal→component mapping: for each newly-plumbed signal (Phase 1 — aviation, AIS disruptions, earthquakes, sanctions, temporal anomalies; Phase 2 — military flights/vessels), which component it scores into and with what sub-formula. The boost list above does not cover signal→component mapping; an unmapped signal gets silently dropped or invented inside Phase 3b.
• The ingest*ForCII side-effect decomposition for Phase 4: enumerate which non-CII side effects (e.g. trackHotspotActivity → hotspotActivityMap) survive the deletion and which feature consumes each. Phase 4's irreversible deletion is gated on this table.

Phase 3b — Server computes the full CII (implementation)

This is the behavior-changing PR. It executes the Phase 3a decision table — it makes no decisions. Implement each canonical formula choice, the scalar tables, the getScoreLevel thresholds, and the proto change. Pair with the Guardrails equality/level/attribution tests below; the PR is not done until the equality test is green.

Phase 3-proto — CiiComponents field naming

Folded into the Phase 3b PR (same cache-key bump); the keep-vs-rename choice is made in Phase 3a. Decide: keep abusing the positional aliases (militaryActivity≡security, …), or rename the .proto fields to unrest/conflict/security/information and regenerate src/generated/server/worldmonitor/intelligence/v1/service_server.ts + src/generated/client/worldmonitor/intelligence/v1/service_client.ts. Either way the CiiScore shape changes → mandatory bump of RISK_CACHE_KEY (v2→v3) propagated to every reader listed at get-risk-scores.ts:626-631.

Phase 4 — Frontend becomes a renderer

• Repoint every CII consumer to cached-risk-scores.ts — both calculateCII() and getCountryScore() consumers. The rg 'calculateCII|getCountryScore' src/ sweep is authoritative and must be run before starting; the known sites are: country-intel.ts:218,740, data-loader.ts:348, search-manager.ts:710, CIIPanel.ts:134, story-data.ts:65, src/services/cross-module-integration.ts:561,630, the internal country-instability.ts:1038 (getTopUnstableCountries), and the getCountryScore path: DeckGLMap.ts / Map.ts (via setCIIGetter), InsightsPanel.ts, military-surge.ts.
• search-manager.ts:710 (panelScores.length > 0 ? panelScores : calculateCII()): the fallback arm is removed, the line keeps the panelScores arm. It is not "converted to a proto consumer" — panelScores already is one.
• Delete calculateCII, getCountryScore, calc*Score, calcNewsConflictFloor.
• Surgical: the ingest*ForCII functions have non-CII side effects (see Risk 1) — keep the ingestion, delete only the CII scoring state. Use the side-effect decomposition table produced in Phase 3a; do not leave it to per-engineer judgment.

Guardrails (not a standalone phase — each test ships in the phase noted)

These are not an independently shippable phase; each test lands inside an earlier phase's PR.

• Equality test (lands with Phase 3b): feed identical fixture component-input signals to the server formulas and the frontend calc*Score functions; assert byte-identical component scores. This is formula parity only — a green equality test is necessary but not sufficient for UI parity (see Principle).
• Level-parity test (lands with Phase 3b): assert the rendered level badge matches across engines for the same score, gated on the Phase 3a getScoreLevel reconciliation. Identical component scores still render different badges until thresholds are unified — the equality test does not catch this.
• Attribution-parity test (lands with Phase 3b, one-shot while both engines coexist): feed real coordinates (straits, borders) through both the server bbox and frontend polygon attribution; assert the country assignment matches, or document the accepted deltas. The equality test alone is blind to this (Risk 2). The frontend attribution is deleted in Phase 4, so this is a transition check, not an ongoing guardrail.
• Standing attribution regression test (permanent — survives Phase 4): a fixed coordinate-set → expected-country fixture for the server's geoToCountry, independent of the frontend. The one-shot attribution-parity test validates the port; this guards against future bbox drift once the frontend polygon attribution is deleted.
• Source-grep test (lands with Phase 4): fails if calculateCII / getCountryScore / calc*Score reappear in src/.

Principle

The server CII and the frontend CII are one metric — after Phase 4 only the server engine exists, so equality is structural. During the transition (Phases 1–3) "equal" means formula parity given identical inputs, verified by the Guardrails equality test. Full production equality additionally requires input parity and attribution parity, which the Guardrails attribution-parity test covers. Note: byte-equality cannot be asserted against the current server information block — it intentionally diverged (#3739); Phase 3a resolves which side is canonical before the equality test can pass.

Risks / gotchas

1. ingest*ForCII have non-CII side effects. ingestMilitaryForCII, ingestProtestsForCII, ingestConflictsForCII call trackHotspotActivity (country-instability.ts:258,270,424,444), feeding hotspotActivityMap consumed by getHotspotBoost. Phase 4 deletes scoring, not ingestion — decompose each function.
2. Formula parity needs input + attribution parity. Porting calcSecurityScore byte-for-byte while feeding differently-attributed arrays still drifts. The Guardrails attribution-parity test guards this; the equality test alone does not.
3. Cold-cache failure mode. On a cold Redis cache + upstream failure, the server returns 31 countries scored on baseline only (get-risk-scores.ts:694-700: computeCIIScores([], emptyAux)). Today the frontend fallback masks this by recomputing from live in-browser streams. Phase 4 removes that mask. Decision required before Phase 4: either keep a thin client-side fallback for the empty-result case, or accept a degraded baseline-only CII on cold start. Do not delete the fallback silently.
4. cross-module-integration.ts is stateful. checkCIIChanges() (line 561) keeps alert history in a module-level previousCIIScores Map and gates on isInLearningMode() (frontend-only state). Moving to a 600 s server cadence changes delta-detection semantics (two snapshots 10 min apart, not live ticks). This is a behavior change, not just an availability concern — re-validate against the learning-mode decision in Phase 3a.
5. Cache key bump is mandatory, not conditional — see Phase 3-proto.
6. Country-set shrink — see "Country-set divergence"; decide in Phase 3a.

Sequencing notes

• Phase 0 ships today; safe given the Scope-boundary commitment.
• Phases 1–2 are additive (signals gathered, not scored) — safe, provided no proto change ships with them (it ships with Phase 3b).
• Phase 3a is a design note (decision table), not code; it is authored after Phases 1–2 land so the vessel-weight check has real data, and its cited formula line numbers are re-verified against current source at the start of Phase 3b. Phase 3b (executing that table, + Phase 3-proto + the Guardrails equality/level/attribution tests) is one PR — the behavior-changing one. It is not done until the equality test is green and every per-component canonical decision from 3a is recorded.
• Phase 4 is the deletion PR — largest blast radius. Gate it on the Risk 3 (cold-cache), Risk 4 (alert retuning), and Risk 6 (country-set) items being resolved and signed off.
• Revertability: Phases 0–3 revert cleanly. Phase 4 deletes code — reverting it means restoring the deleted engine and the ingest*ForCII decomposition. Treat Phase 4 as the point of no return.

Deferred / Open Questions

From 2026-05-22 review

• Cold-cache fallback decision (Risk 3). Phase 4 deletes the frontend engine that today masks the cold-cache failure mode — on cold Redis + upstream failure the server returns 31 baseline-only scores (get-risk-scores.ts:694-700). Decide before Phase 4: keep a thin client-side fallback for the empty-result case only, or accept a degraded baseline-only CII on cold start (and decide whether a degraded score needs a UI staleness indicator). This is an architecture/UX call that depends on tolerance for a cold-start baseline-only CII — it cannot be auto-resolved. Blocks Phase 4. (adversarial, product-lens)