docs/06_explainability.md

# 06 — Explainability and mechanistic validation (RQ5)

[← Back to 05: Decision framework](05_decision_framework.md) | [README](../README.md)

Covers **Phase 9**. Every number is generated into
[`outputs/reports/`](../outputs/reports/).

**RQ5: do the learned attributions recover known electrochemical degradation
signatures, so a process engineer can trust and act on them?** Answer: partly —
two hypotheses supported cleanly, one supported *but for the wrong physical
reason*, and one not supported at all.

---

## 1. Hypothesis verdicts

Four falsifiable hypotheses, stated in advance. **Agreement was not forced.**

| | Hypothesis | Verdict |
|---|---|---|
| **H1** | ΔQ(V) variance features rank top-3 by SHAP at budget ≥ 50 | **SUPPORTED** |
| **H2** | Resistance features dominate at very low budgets (5–10) | **NOT SUPPORTED** |
| **H3** | Thermal SHAP is monotonic in the thermal feature value | **SUPPORTED**, with a mechanism caveat that changes its meaning |
| **H4** | Without recipe descriptors, curve features absorb the attribution | **SUPPORTED** |

### H1 — SUPPORTED

A ΔQ(V) variance feature sits in the top 3 at **every** budget ≥ 50:
**rank 2 at N=50, rank 1 at N=100**, where it carries 27.2% of total
attribution.

This is the expected result and it is consistent with Gate 1, which reproduced
the published log₁₀ var(ΔQ(V)) relationship at R² = 0.859. The model is leaning
hardest on the signal the literature says it should.

### H2 — NOT SUPPORTED

The prediction was that resistance features would carry disproportionate weight
at 5–10 cycles, consistent with formation-stage diagnostics. They do not:

| | Resistance share of total attribution |
|---|---|
| Budgets 5–10 | **5.3%** |
| Budgets 50–100 | **7.1%** |

The share is *lower* early, not higher — the opposite of the prediction. Two
readings are possible and this project cannot distinguish them: either the
resistance measurement in this corpus is too coarse to carry formation-stage
information (it is a single per-cycle scalar, not an impedance spectrum), or the
curve-shape features already capture that information more efficiently at every
budget. Reported as a clean negative.

### H3 — SUPPORTED, but the mechanism is not thermal

All **4 of 4** thermal features show |Spearman| > 0.7 between feature value and
SHAP contribution: the model uses them consistently and directionally, which is
what the hypothesis stated.

**But the physical reading is wrong, and the check that found this was run
because the result contradicted an earlier measurement.** Phase 3 measured
thermal signal as weak (|ρ| ≈ 0.22–0.25); Phase 9 found `thermal_exposure`
carrying **23.2% of attribution as the #2 feature**. That tension was
investigated rather than explained away:

| Quantity | Spearman vs log₁₀ cycle life | Spread across cells |
|---|---:|---:|
| `thermal_exposure` (∫T dt) | **−0.748** | **2.56×** |
| `temp_mean` (per-cycle mean temperature) | +0.222 | 1.21× |

The two phases measured **different quantities**. Phase 3's was a per-cycle mean
temperature; Phase 4's `thermal_exposure` integrates temperature over **time**.
In a 30 °C-controlled chamber the temperature term barely varies (1.21× across
the whole cohort), so **most of the integral's variation is cycle duration**,
and it correlates 0.414 with mean charge time.

**Conclusion: `thermal_exposure` is predictive as a time-under-load proxy, not
as an Arrhenius thermal effect.** The hypothesis as stated passes; the physical
interpretation it was meant to support does not. This is exactly the failure
mode the module docstring warned SHAP cannot detect on its own — *SHAP reports
what the model used, not whether the mechanism is real* — and it took an
external consistency check to surface it.

The plain-language audit sheet reflects this: `thermal_exposure` is described to
the engineer as "cumulative time the cell spent under load", with an explicit
note that it should not be read as a heating effect.

### H4 — SUPPORTED

Removing the Group F process-recipe descriptors:

| | With recipe | Without recipe |
|---|---:|---:|
| Curve-feature share of attribution | 41.1% | **47.2%** |
| Cross-validated RMSE (log₁₀) | 0.0597 | 0.0597 |

Curve features absorb the attribution **and** the information — RMSE is
unchanged to four decimal places over 10 grouped folds. This is the favourable
outcome for RQ4: the model is not merely memorising a recipe-to-lifetime lookup,
because removing the recipe costs it nothing.

The failure mode this hypothesis exists to detect would have been a rise in
curve attribution *with* degraded performance — attribution moving without
information moving. That did not happen.

## 2. Does the anomalous cell dominate the attributions?

Phase 4 documented **b1c41**'s `dq_kurtosis` at 112.9 against a cohort IQR of
[−1.10, −0.30] — roughly 140 robust-z from the median. A single cell that
extreme can dominate a global ranking, so every global result was computed
**with and without it**, and it was never silently excluded.

**It does not dominate.** At budget 100, removing b1c41:

- the **top 7 features are unchanged** in rank
- the **maximum rank shift** anywhere in the top 10 is **2 places**
- the largest share change is **0.011** (`ir_at_cycle_2`, 0.055 → 0.044)

The concern was reasonable and the measurement settles it. Both rankings are in
[`shap_anomaly_comparison.csv`](../outputs/reports/shap_anomaly_comparison.csv).

## 3. Global attribution at the reference budget

| Rank | Feature | Share |
|---:|---|---:|
| 1 | `dq_var` | 27.2% |
| 2 | `thermal_exposure` | 23.2% *(read as time-under-load — see H3)* |
| 3 | `q_discharge_curve_area_N` | 12.2% |
| 4 | `cc_time_fraction` | 8.4% |
| 5 | `recipe_stress_x_thermal` | 6.7% |
| 6 | `ir_at_cycle_2` | 5.5% |
| 7 | `policy_c_mean` | 4.4% |

Figures: `fig18` (global importance), `fig19` (beeswarm), `fig20` (importance vs
budget), `fig21` (per-cell waterfalls), `fig22` (verdict summary).

## 4. The QC audit sheet

The artifact that makes this work concrete rather than decorative: a one-page
sheet per cell, in
[`outputs/reports/audit_sheets/`](../outputs/reports/audit_sheets/).

Each sheet carries the prediction and its conformal interval, the assigned
grade, **the expected cost of every available action** (so the reader sees what
the alternative would have cost rather than being told the answer), the top-5
drivers in plain language, and any data-quality flags inherited from Phase 2.

**A defect found and fixed while building it.** The first version quoted
feature values straight from the model pipeline — which ends in a
`StandardScaler`, so the sheet was showing **z-scores**: "how much the discharge
curve's shape changed: −0.8224". That is meaningless to the exact reader the
artifact exists for. Sheets now quote the **physical value alongside the cohort
median**, so the engineer sees "this cell: 0.000433 (typical cell: 0.00017 —
this one is above typical)".

An extract:

```
     1. How much the discharge curve's shape changed between the early
        and late reference cycles
        this cell: 0.000433   (typical cell: 0.0001696 -- above typical)
        effect on the decision: LOWERED the predicted cycle life
        for reference, a HIGH value means a large change, which indicates
        faster loss of usable lithium
```

## 5. Limitations

- **SHAP explains what the model used, not whether the mechanism is real.** H3
  is the concrete demonstration: a hypothesis can pass while the physical story
  behind it is wrong. Every verdict here is evidence about consistency, not
  proof of mechanism.
- **The audit sheet's usefulness has not been tested with a process engineer.**
  The tests check completeness and internal consistency, which is necessary and
  not sufficient. Only a human reader can judge whether the explanation lands.
- Attribution is computed on a model fitted to the full cohort, which is correct
  for "what did the deployed model use" and would be wrong for any performance
  claim. No performance number is derived from that fit.

---

[← Decision framework](05_decision_framework.md) · [README](../README.md) · [Protocol shift →](07_protocol_shift.md)