cascade_risk / knowledge /expert_predict.md
Lucasoppem's picture
Sync from GitHub main (part 2)
36f9d47 verified
|
Raw
History Blame Contribute Delete
14.6 kB
<!--
Maintainer notes:
- This file defines HOW to predict; the 12-domain taxonomy and severity
thresholds themselves live in knowledge/expert_extraction.md.
When the taxonomy changes, update Β§Scope Alignment with Extraction
here and the 12-label list mirrored there.
- Things owned by this file (change here only):
(1) Region Similarity Factors (climate zone, terrain, urbanization)
(2) Common Cascade Patterns library (the 4 patterns + any you add)
(3) Prediction Principles
(4) BFS Reasoning Paradigm (v0.2 layer-by-layer prediction)
(5) STOP Signal Contract (v0.2 iterative prompt's `stop_reason`)
(6) Layer Consistency (time monotonicity, severity transitions, no re-derivation)
- When extending to non-European data (e.g. Asian floods), add to
Β§Region Similarity Factors and Β§Common Cascade Patterns. See
technical_report/data_pipeline/cascade_extraction_report.md Β§6.5.3 for the checklist.
-->
# Expert Knowledge: Cascade Risk Prediction
## Region Similarity Factors
When predicting cascades for a new event, consider these factors for matching with historical events:
### Geographic Similarity
- **Climate zone**: Mediterranean, Atlantic, Continental, Nordic β€” determines flood characteristics
- **Terrain**: Coastal, river basin, mountain, urban plain β€” affects flood behavior
- **Urbanization**: Dense urban areas have more infrastructure interdependency
### Infrastructure Similarity
- **Infrastructure age and density**: Older European cities have underground systems vulnerable to flooding
- **Power grid topology**: Centralized vs. distributed generation affects cascade risk
- **Flood defense maturity**: Netherlands' defenses differ greatly from Southern European ones
- **Healthcare density**: Hospital beds per capita, distance to alternative facilities
### Event Similarity
- **Flood type**: River flood, flash flood, coastal surge, pluvial flood β€” different cascade profiles
- **Severity magnitude**: Damage in USD, affected population, area covered
- **Duration**: Short intense floods vs. prolonged flooding produce different cascades
- **Season**: Winter floods compound with heating/energy demand; summer floods affect tourism/agriculture
## Common Cascade Patterns in European Floods
> **Arrow ↔ `parent_ids` mapping (v0.2 issue #9 / B').**
>
> In the patterns below, `A β†’ B β†’ C` means B was caused by A, and C by B. When you emit the JSON:
> - A is a first-order flood effect β†’ `A.parent_ids = []`
> - B is caused by A β†’ `B.parent_ids = ["<A's id>"]`
> - C is caused by B β†’ `C.parent_ids = ["<B's id>"]`
> - C.parent_ids must NOT also contain A β€” A is a grandparent of C via B (no-grandparent rule).
>
> **Multi-parent only when truly joint:** if a node D is caused by A and B *together* (and B is not an ancestor of A or vice versa), then `D.parent_ids = ["<A's id>", "<B's id>"]`. Apply the ablation test: removing either A or B must change D's description, severity, or timing.
### Pattern 1: Infrastructure Cascade
`Flood β†’ Power outage β†’ Communication failure β†’ Delayed emergency response β†’ Increased casualties`
- Most common in dense urban areas
- Power and communication failures often co-occur because cell-tower backup batteries discharge after the grid goes down β€” describe the mutual amplification in the `mechanism` field; do NOT use a feedback-loop field (no longer in the schema).
### Pattern 2: Water Contamination Cascade
`Flood β†’ Sewage overflow β†’ Drinking water contamination β†’ Waterborne disease β†’ Hospital overload`
- Common in areas with combined sewer systems (older European cities)
- Time delay of 3-7 days before disease onset
### Pattern 3: Transport Isolation Cascade
`Flood β†’ Road/rail disruption β†’ Community isolation β†’ Supply shortage β†’ Vulnerable population at risk`
- Common in rural/mountain areas with limited alternative routes
- Severity amplifies for elderly/disabled populations
### Pattern 4: Economic Cascade
`Flood β†’ Business property damage β†’ Business closure β†’ Unemployment β†’ Long-term economic decline`
- More severe in areas dependent on single industries
- Insurance coverage varies significantly across European regions
## Prediction Principles
1. **Start with the most likely cascades** based on historical patterns in similar regions
2. **Adjust for local context** β€” infrastructure age, population density, season, preparedness level
3. **Mutual-amplification patterns** (e.g. power ↔ comms) are critical to surface β€” describe the bidirectional mechanism in the `mechanism` field of the relevant nodes (the schema does not encode loops as edges; use forward edges + clear mechanism prose)
4. **Assign realistic time offsets** based on historical data from similar events
5. **Include confidence scores** β€” be honest about uncertainty, especially for rare cascade paths
6. **Cite reference events** β€” explain which historical events informed each prediction
7. **Default to single-parent** β€” most BFS-emitted nodes have exactly one parent (the frontier node they were expanded from). Multi-parent is a deliberate declaration of joint necessity; apply the ablation + no-grandparent rules.
## BFS Reasoning Paradigm (v0.2)
Starting in v0.2, prediction does **not** produce the whole DAG in a single LLM call. Instead the
DAG is grown layer by layer using breadth-first search, with one LLM call per layer:
- **Layer 0 (initial call)** β€” produces only the **first-level cascades**, i.e. nodes whose
`parent_ids = []` because they are direct effects of the flood itself. The retrieval side
hands the model the top-`initial_top_k` historical *first-level* edges as seed evidence.
Prompt template: `prompts/predict_initial.txt`.
- **Layer β‰₯1 (iterative call)** β€” the previous layer's nodes are the **frontier**. For each
frontier node, the retriever returns the top historical edges whose `parent_text` matches that
node's description. The model emits **only the next layer** of cascades, anchoring each new
node's `parent_ids` to the frontier (or, for multi-parent joins, to any earlier DAG node).
Prompt template: `prompts/predict_iterative.txt`.
**Why BFS instead of one-shot.** v0.1 dumped the full retrieved chains into a single prompt and
asked for the whole DAG in one shot. The model often collapsed the result into a "star DAG"
(every node's `parent_ids = []`). BFS forces explicit layer-by-layer reasoning and makes the
parent linkage rule mechanically enforceable per turn (the validator only accepts new parents
from the current frontier βˆͺ DAG snapshot).
**Implications for this expert knowledge file.**
- `Common Cascade Patterns` arrows (`A β†’ B β†’ C`) still encode the same DAG topology, but the
model will encounter them across **multiple turns**: A in layer 0, B in layer 1, C in layer 2.
Whatever turn the model is on, only the immediate next hop matters that turn β€” do not try to
unfold the whole pattern into a single iterative response.
- Region similarity factors apply at every layer; iterative calls receive the new event's
metadata in `event_inputs` so the model can keep adjusting for local context.
## STOP Signal Contract (v0.2)
The iterative prompt's output JSON carries a `stop_reason` field that controls BFS termination
at the model layer. Valid values:
- `null` β€” the layer is non-empty and BFS should continue.
- `"saturation"` β€” there is no meaningful next-hop for any frontier node (e.g. the cascade has
reached a stable end-state like long-term economic decline, or all natural follow-ons are
already in the DAG snapshot). The layer **MUST be empty** when this is set.
- `"out_of_domain"` β€” the retrieved evidence is from events too dissimilar to the new event
(wrong region, wrong flood type, wrong severity) to support a useful prediction. Again, the
layer **MUST be empty** when this is set. Prefer this over forcing speculative cascades.
When to return empty + non-null `stop_reason`:
- All frontier nodes have already been extended in earlier layers (no new edges to propose).
- Every retrieved edge points to a cascade already present in `dag_snapshot`.
- The iterative call's confidence in any new node would be < ~0.4.
The orchestration layer also enforces non-semantic stops independently of the model: layer
budget (`max_layers`), total node budget (`max_total_nodes`), retrieval similarity floor
(`similarity_threshold`), and the absolute time window (`time_window_hours`). The model's
`stop_reason` is a *semantic* stop signal, complementary to those budgets.
## Layer Consistency
Per-layer outputs must remain coherent with the DAG already produced. The following invariants
hold across BFS layers and are enforced by the validator on every iterative response:
- **Time monotonicity** β€” every new node satisfies
`time_offset_hours β‰₯ max(parent.time_offset_hours for parent in parent_ids)`.
In practice, use the historical `time_offset_hours_delta` from the retrieved edge as the
expected gap between parent and child, and add it to the parent's absolute time.
- **Severity transitions** β€” severity is allowed to rise or fall, but large jumps
(e.g. `low β†’ critical`) require a `mechanism` that explicitly names the compounding factor
(mutual amplification with another cascade, vulnerable population, infrastructure failure).
Default expectation: severity attenuates downstream.
- **No re-derivation** β€” do not re-emit a cascade that already exists in `dag_snapshot`. Use
`parent_ids` to link to the existing node instead.
- **No same-layer causal links** β€” siblings within one layer share parent(s); none of them
causes another. If `mechanism` reads "E_n leads to E_m" where both are in the same layer,
one of them belongs to a later layer.
- **Closed-domain taxonomy and severity rubric** β€” unchanged from v0.1; both layer-0 and
layer-β‰₯1 outputs must use the 12-label domain taxonomy and the four-level severity scale
defined in Β§Scope Alignment with Extraction.
## Scope Alignment with Extraction
The extraction pipeline produces a closed-taxonomy cascade graph:
- **Budget:** at most 40 nodes per event
- **Time window:** all cascades within 2 weeks (time_offset_hours ≀ 336)
- **Closed domain taxonomy** β€” predictions MUST use exactly one of these 12 labels:
`infrastructure/power`, `infrastructure/water`, `infrastructure/transport`, `infrastructure/communication`,
`health/casualties`, `health/hospital_service`, `health/disease_outbreak`,
`social/evacuation`, `social/supply_shortage`,
`economy/business_damage`, `economy/agriculture`,
`environment/contamination`
- **Severity thresholds** β€” critical / high / medium / low per the same quantitative rubric as extraction (see `expert_extraction.md` for the thresholds)
Keeping predict and extract aligned is what lets RAG retrieval find relevant historical chains β€” mismatched domains or severity scales silently break similarity matching.
## Description Style Guidance (v0.2 issue #11)
Gold cascade chains (extracted from news in `data/processed/cascade_chains/*.json`) are dense with **named entities** β€” specific places, numbers, institutions. Real examples from the new gold:
- `"Approximately 42,000 customers across 32 villages and towns lost power; suburbs left without electricity"` (2025-0848-UKR E8)
- `"At least 219 deaths reported (211 in Valencia, 7 in Castilla-La Mancha, 1 in Andalusia)"` (2024-0796-ESP E2)
- `"Winds up to 128mph were recorded in Pointe du Raz, Brittany"` (2023-0734-FRA E8)
Predictions evaluated by cosine similarity against gold need to carry comparable named-entity density to score, even when the underlying causal reasoning is correct. Generic phrasing ("widespread X", "localized Y") embeds in a separate region of the cosine space from named-entity-rich gold.
**Rules** (mirrored in `prompts/predict_iterative.txt` and `prompts/predict_initial.txt`):
1. **Echo named entities from the new event description.** The event's `description` (passed in `event_inputs`) typically names cities, regions, fatality counts, affected populations, named infrastructure. Use those names verbatim when the cascade you're describing applies to them.
2. **Use historical / retrieved edges as patterns, not as named-entity sources.** A retrieved edge that says "substation in Marseille flooded" tells you "substation flooding β†’ outage" is a known pattern; do NOT carry "Marseille" into a prediction for an Odesa flood. Take the cascade type, re-instantiate with the new event's named places.
3. **Numbers only when quantitatively supported.** If the event description says "9 deaths, 2,000 affected", you can write "~2,000 households impacted" or "9 fatalities reported". If the event description gives no number, write the cascade without one β€” do NOT fabricate.
4. **Mechanism field carries the physical / operational link.** "Substation transformers submerged by floodwater trigger automatic shutoff" beats "loss of mains power".
5. **Anti-patterns to avoid**:
- "widespread", "localized", "various impacts" β€” empty quantifiers
- "disruption to services" β€” no concrete cascade
- Named entity (city, hospital, road) that is in NEITHER the event description NOR a retrieved edge as a *pattern element you can legitimately re-instantiate* β€” fabrication, treated the same as a hallucinated cascade
This guidance inherits the **grounding rule** from `knowledge/expert_extraction.md` (v0.2 issue #10): hallucinated content is worse than missing content. When in doubt, prefer the generic phrasing over a fabricated named entity.
## Evidence reading rules (v0.5 issue B+C)
Each retrieved evidence is a structured tuple:
```
(parent_domain "redacted text") β†’ (child_domain "redacted text"),
severity=A→B, +Xh
```
`<N>` and `<LOC>` are deliberate placeholders β€” they hide specific
quantities and places from the historical edge so you treat the edge
as a **causal pattern**, not a verbatim template.
Apply the pattern to **this event's** specifics:
- The new event's description is your only source of grounded numbers
and place names.
- If the description doesn't mention "Italy" or "10K households",
neither should your predictions.
- A mechanism phrase like "households lose power" is structural and
may be reused with this event's geography substituted in (or omitted
if no specifics are available).
**When in doubt: omit.** A grounded generic description scores under cosine β‰₯ 0.35; a fabricated specific shifts the embedding into unrelated regions and destroys the eval signal.