| --- |
| tags: |
| - causal-inference |
| - time-series |
| - foundation-model |
| - temporal-causal |
| - causal-judgment |
| license: other |
| license_name: proprietary |
| license_link: LICENSE |
| --- |
| |
| # TCPFN — Temporal Causal Prior-Data Fitted Networks |
|
|
| A family of causal reasoning foundation models — predict effects, judge trustworthiness, operate zero-shot. Three checkpoints share one architecture (12-layer transformer, `embed_dim=512`, 8 heads, HL-Gaussian output head) and differ only in training-data distribution and curriculum. |
|
|
| ## Pick by task |
|
|
| | Task | Best checkpoint | Path | |
| |------|-----------------|------| |
| | General causal discovery (biology, cross-sectional, short-lag) | **v2.1** | `models/temporal/final.pt` | |
| | Industrial / long-range discovery (12+ h lags, digester→paper machine etc.) | **v2.2** | `models/v2.2/final.pt` | |
| | Effect estimation (CATE / PEHE) | **v3** | `models/v3/final.pt` | |
|
|
| All three are zero-shot. Pick the one matching your task — specialisation beats generalist on every task we've measured. |
|
|
| ## Shared contributions |
| 1. **Temporal Token Design** — first PFN for temporal panel data. |
| 2. **Causal Judgment Head** — learned reliability signals (null detection, regime classification, identifiability, mediation, confounding). |
| 3. **Causal Regime Prior** — direct / confounded / mediated / feedback training structures. |
| 4. **Self-Calibration** — auto-detects natural experiments in sensor data. |
| 5. **End-to-End System** — discovery + estimation + judgment + RCA from one forward pass. |
|
|
| ## Shared capabilities |
| - **Causal Discovery** — pairwise interventional CATE with judgment-aware edge scoring, natural-experiment detection, continuous treatment, multi-lag estimation, asymmetry penalty. |
| - **Effect Estimation** — temporal CATE trajectories with distributional output. |
| - **Causal Judgment** — null-effect detection, regime classification (learned heuristics, not formal guarantees). |
| - **Root Cause Analysis** — 8-method ensemble (AERCA, ESD, ProRCA, GCM noise, ICC, Shapley, counterfactual, chain tracing). |
|
|
| --- |
|
|
| ## v2.1 — default discovery model |
| - 200K steps, curriculum-trained (Phase 1 CATE-only → Phase 2 +Null → Phase 3 Full). |
| - Mixed prior: 40% CausalTimePrior + 30% base + 30% CausalFM. |
| - Training window: `max_T_pre=50, max_T_post=30`. |
| - Hardware: RTX 5090, ~4.1 h, 13.9 steps/s. |
|
|
| ### Discovery benchmarks (14 datasets, 6 domains, zero-shot) |
| - Sachs (11 proteins, biological): F1 0.412, **AUROC 0.725** (vs Granger 0.291 / 0.621) — **champion**. |
| - Causal Rivers (environmental): F1 0.319, AUROC 0.955. |
| - Tennessee Eastman (52 vars, industrial): F1 0.314, AUROC 0.904. |
| - SWaT (51 vars, water treatment): F1 0.265, AUROC 0.859. |
| - CauseMe NVAR-5 / NVAR-10: F1 0.571 / 0.439. |
| - Highest default-threshold F1 on 6 of 14 datasets. |
| - Hallucination FPR: 0.02–0.08 (down from 1.0 in v2.0). |
|
|
| ### Training metrics (mean over steps 150K–200K) |
| - EffectLoss ~2.9 | JudgmentLoss ~2.8 |
| - NullF1 0.94 | NullAUROC 0.99 | NullBrier 0.04 | NullSep 0.86 |
| - RegimeAcc 0.68 | RegimeMacroF1 0.48 |
|
|
| ### Limitations |
| - CATE estimation weak (PEHE 0.92) due to per-group Z-standardisation — **use v3 for estimation**. |
|
|
| --- |
|
|
| ## v2.2 — industrial / long-range specialist |
| Built for 12+ hour causal lags in industrial control loops (digester → paper machine, reactor → downstream controller). Training window extended 4× and curriculum rebalanced to include null-effect batches in Phase 2. |
|
|
| - 200K steps, BF16 mixed precision, `head_lr_scale=0.1` (decouples output-head learning from backbone to prevent late-stage drift collapse). |
| - Training window: `max_T_pre=200, max_T_post=100, max_horizon=500` — supports lags up to ~16 h at 2-min sampling. |
| - Manual NaN-skip with observability (saves first NaN-producing batch, aborts if skip rate ≥ threshold). |
| - Hardware: RTX 5090, ~14.9 h. |
|
|
| ### Discovery benchmarks (default threshold 0.5) |
| Strong on industrial / multivariate temporal data — **use this** when lags exceed ~1 h or when data is genuinely time-series (not stitched cross-sectional). |
|
|
| | Dataset | Default F1 | Best F1 | AUROC | |
| |---------|-----------|---------|-------| |
| | Tennessee Eastman | **0.512** | 0.545 | **0.972** | |
| | SWaT | **0.463** | 0.552 | **0.945** | |
| | CauseMe VAR-5 | 0.769 | 0.800 | 0.960 | |
| | CauseMe NVAR-5 | 0.800 | 0.800 | 0.863 | |
| | CauseMe VAR-10 | 0.488 | 0.643 | 0.812 | |
| | CauseMe NVAR-10 | 0.634 | 0.634 | 0.759 | |
| | CauseMe Lorenz96-10 | 0.484 | 0.638 | 0.699 | |
| | Sachs | 0.174 | 0.308 | 0.565 | |
|
|
| Granger and PCMCI collapse on industrial data — they over-predict (1897 edges on TE vs 38 true), giving F1 ~0.04. TCPFN v2.2 is the only method with usable precision + recall together. |
|
|
| ### Estimation benchmarks |
| - Overall PEHE 0.917 | ATE MAE 0.504 | trajectory correlation ≈ 0 — **use v3 for CATE**. |
|
|
| ### Limitations |
| - **Sachs regressed** vs v2.1 (AUROC 0.565 vs 0.725). Use v2.1 for cross-sectional biological graphs. |
| - Estimation degraded — trades short-range precision for long-range reach (see scar-tissue entry L-33 in project docs). |
|
|
| --- |
|
|
| ## v3 — estimation champion (experimental) |
| Tag: `3.0.0-exp-global-std`. Global standardisation fix for the per-group Z-score bias that caps v2.1/v2.2 estimation quality. |
|
|
| - 200K steps. |
| - **PEHE 0.72** (vs v2.1 0.92 and v2.2 0.92) — best of the three on CATE estimation. |
| - Discovery regressed slightly as trade-off; not yet benchmarked across all 14 discovery datasets — **use v2.1 or v2.2 for discovery**. |
|
|
| ### Limitations |
| - Experimental tag — standardisation change not yet battle-tested beyond estimation. |
| - Full benchmark matrix still pending. |
|
|
| --- |
|
|
| ## Usage |
|
|
| ```python |
| from tcpfn import TemporalCausalAnalyzer |
| |
| # Discovery on general data (biology, cross-sectional) |
| analyzer = TemporalCausalAnalyzer(temporal_model="models/temporal/final.pt") |
| |
| # Industrial / long-range discovery (lags in hours) |
| analyzer = TemporalCausalAnalyzer(temporal_model="models/v2.2/final.pt") |
| |
| # Effect estimation (CATE trajectories, PEHE-sensitive work) |
| analyzer = TemporalCausalAnalyzer(temporal_model="models/v3/final.pt") |
| |
| report = analyzer.run("sensor_data.csv") |
| print(report.edges) # causal graph with edge strengths and lags |
| print(report.summary()) # human-readable summary |
| |
| result = analyzer.explain_event( |
| data_path="sensor_data.csv", |
| target_var="temperature_sensor", |
| event_time="2025-11-15 14:15", |
| ) |
| print(result.summary()) # ranked root causes + causal chains |
| ``` |
|
|
| ## Cross-cutting limitations |
| - Regime classification is noisy (~0.68 accuracy, high eval variance). Judgment heads are **learned heuristics**, not formal guarantees. |
| - Low-dim cross-sectional data stitched into pseudo-timeseries is out-of-distribution for v2.2 and v3; use v2.1. |
| - v3 has not yet been run on the full discovery benchmark suite. |
|
|
| ## Paper |
| Stalupula et al., "Temporal Causal Prior-Data Fitted Networks for Panel Data with Learned Reliability Signals" |
|
|
