RESEARCH PREVIEW · APA

Aperture Audit

A credit-decision dashboard where every prediction ships with an exact, closed-form SHAP-style attribution — cross-checked against an independently derived LIME explanation built from scratch, not just asserted.

Launch live demo Read the code

HEADLINE RESULT

Two independently-derived explanation methods — exact SHAP attribution and a from-scratch LIME local surrogate — agreed on the direction of every feature's effect in 100% of a 140-decision cross-check (20 applicants × 7 features).

100%

SHAP / LIME sign agreement across 140 feature-decisions checked

credit features attributed per decision, ranked by effect size

0.99

average LIME local surrogate fidelity (R²)

independently-derived explanation methods per decision

METHOD

Two explanations, cross-checked, not just one asserted

Logistic regression is chosen deliberately: it's the case where SHAP attribution has an exact closed form, making it possible to verify the second, sampling-based method against it.

Train

A logistic regression credit model is trained on synthetic applications with a known ground-truth generating function.

Attribute (SHAP)

Each feature's exact contribution to a decision is computed in closed form: coefficient × (value − population mean).

Attribute (LIME)

Independently, 400 small random perturbations around the applicant are run back through the real model, and a local weighted regression recovers a second explanation with zero knowledge of the model's internals.

Cross-check

The two explanations are compared feature by feature, on direction and magnitude, using the same additive formula for both.

Narrate

The top 3 drivers of each decision are surfaced in plain language — the level of detail EU AI Act-style regulations require for high-stakes automated decisions.

FINDINGS

What the synthetic test runs showed

An initial version of the cross-check compared raw LIME coefficient signs against SHAP's deviation-adjusted contributions — an apples-to-oranges mismatch that produced only 2/7 agreement. Fixing the comparison to use the same formula for both raised agreement to 100%.

Recovered model coefficients matched the sign of the synthetic data's true generating weights on all 7 features (e.g. late payments and debt-to-income both correctly learned as negative effects on approval).

LIME's local surrogate fidelity stayed above 0.99 R² across sampled applicants, meaning the local linear approximation captures the real model's behavior almost exactly in each applicant's neighborhood.