numerical-provenance

Every numeric output carries its convergence state.

A solver that has not converged returns a plausible-looking float. The float propagates through arithmetic. Eventually a user sees a number. The number is garbage. Nothing in the pipeline flagged it.

Concrete example: a Jacobi eigendecomposition on a 448×448 matrix was given max_iter=200. It needs ~100,000 iterations to converge on that matrix. It returned 97.705. The number propagated into a report. Nothing flagged it.

numerical-provenance fixes this by making the convergence state a first-class property of the number.

The claim in one sentence

Attach convergence metadata to every numeric value at the point of production; propagate it through every operation; refuse to format or consume it unless the caller explicitly opts in.

What it produces

Every solver output is a PValue carrying:

• value — the raw float
• provenance.methods — the chain of solvers and operations that produced it
• provenance.converged — True iff every upstream solver converged
• provenance.max_iterations — worst iteration count in the chain
• provenance.max_residual — worst residual in the chain
• provenance.min_tolerance — tightest tolerance any solver aimed at

Install

pip install numerical-provenance

Or run the module directly:

```bash
python numerical_provenance.py

Pure stdlib. No dependencies.

Usage

Basic

from numerical_provenance import (
    solver, strict, report, assert_converged,
    PValue, Provenance, UnconvergedError,
)

@solver("my_solver", tol=1e-10)
def my_solver(A):
    """Return (value, info)."""
    value, iterations, residual = do_the_work(A)
    info = {
        "converged": residual < 1e-10,
        "iterations": iterations,
        "residual": residual,
    }
    return value, info

pv = my_solver(A)

# Sanctioned reporting.  Refuses unconverged values.
print(report(pv))

# Sanctioned assertion.  Fails loudly on unconverged values.
assert_converged(pv, "startup")

# Arithmetic propagates provenance.
result = (pv + 5.0) / 2.0
print(report(result))

Strict consumers

@strict
def aggregate(a, b):
    """Refuses unconverged inputs."""
    return (a + b) / 2.0

try:
    aggregate(pv_broken, 5.0)
except UnconvergedError as e:
    print(f"refused: {e}")

Arrays

from numerical_provenance import PArray

# A batch of independent sub-solves.
ranks = PArray([my_solver(sub_matrix) for sub_matrix in batches])

# Elementwise arithmetic preserves the mixed state.
doubled = ranks * 2.0 + 1.0

# Reductions merge provenance.
total = ranks.sum()
if total.converged:
    print(f"total = {total.value}")
else:
    print(f"refused: {report(total)}")

Cross-checks

from numerical_provenance import cross_check, Disagreement

result = cross_check([solver_a, solver_b, solver_c], A, tol=1e-6)

if isinstance(result, PValue):
    print(f"all agree: {report(result)}")
else:  # Disagreement
    print(f"disagreement: {report(result)}")
    idx, name, dist = result.outlier()
    print(f"outlier: {name}")

Adaptive budgets

from numerical_provenance import jacobi_budget

# Instead of max_iter=5000 (which fails at n=40):
budget = jacobi_budget(n)  # = max(2000, 15 * n * n)

The six defenses

The module provides six levels of escalation for the same bug class:

level	mechanism	action
1. tag	@solver	every output is a PValue, not a float
2. propagate	arithmetic	taint spreads through + - * / **
3. refuse	report()	refuses to format unconverged values
4. raise	@strict	raises UnconvergedError on unconverged args
5. assert	assert_converged()	AssertionError at pipeline startup
6. bypass	.unwrap(allow_unconverged=True)	requires explicit opt-in

Any pipeline can choose its level. The choice is explicit at every boundary.

Benchmarks

The demo (python numerical_provenance.py)

part	concept	outcome
0	naive pipeline	prints 8.953885 with no flag
1	PValue	carries converged=False, residual=18.1
2	taint propagation	(short + 5) / 2 stays unconverged
3	report()	refuses unconverged, formats converged
4	@strict	raises on short, accepts converged
5	escape hatch	leaves UNCONVERGED tag even when bypassed
6	converged path	1.71092 with iterations=972, residual=3.19e-10
7	assert_converged	fails loudly, returns raw float on success
8	PArray	mixed batch, elementwise arithmetic, reductions
9	adaptive budget	n=10, 20, 30, 40 all converge
10	cross_check	3 cases: agreement, unconverged, tolerance mismatch
11	summary	the discipline, in one place

Total runtime: ~3 seconds on a laptop CPU.

Iteration budget scaling

jacobi_budget(n) = max(2000, 15 * n²)

n	budget	observed iters	residual	margin
10	2000	181	5.61e-11	11×
20	6000	972	3.19e-10	6.2×
30	13500	3422	1.23e-09	3.9×
40	24000	5510	1.64e-09	4.4×

The observed iterations scale roughly as n^2.4. The factor of 15 gives comfortable margin up to n≈100. A hardcoded max_iter=5000 would fail at n=40; the formula does not.

The outlier rules

When cross_check returns a Disagreement, the outlier is identified by three rules in order:

1. Unconverged wins. A solver that did not converge is the outlier by construction, regardless of where its value landed.
2. Loosest tolerance wins. If all solvers converged but tolerances differ, the one that aimed at the loosest tolerance is less trustworthy. Ties break by distance from the tight-group mean.
3. Distance from mean. If all tolerances are equal, fall back to the value farthest from the mean.

The order matters. A rule based purely on distance-from-mean gets the outlier wrong in two cases:

• Two values: the "median" is ambiguous (upper vs lower), so the distance metric biases toward one side.
• A loose solver whose value happens to land near the tight solver's value is not flagged, even though it is less trustworthy.

Rules 1 and 2 fix both cases. Rule 3 is the fallback for the case where every solver ran with the same tolerance and still disagreed — which means the disagreement is real, not a tolerance artifact.

When to use it

• Any numerical pipeline with iterative solvers. Eigenvalue solvers, optimizers, MCMC samplers, fixed-point iterations, root-finders. If a solver has a convergence criterion, wrap it.
• Multi-solver pipelines. When two independent methods should agree, cross_check makes disagreement an output rather than a silent pass-through.
• Publication-grade computation. A number that goes into a paper should carry its convergence state. report() refuses to format anything else.
• Long-running scientific pipelines. A single unconverged solve at hour 6 of a 24-hour run is the kind of thing that silently poisons the entire result. assert_converged at every stage boundary catches it.

When not to use it

• When the solver has no convergence criterion. A direct matrix inverse, a closed-form formula, a table lookup — these have no iteration count. Wrap them with a plain PValue(v, _PLAIN_PROV) or accept the raw float.
• When performance is critical and profiling shows the wrapper dominates. The PValue layer is cheap (a few microseconds per operation), but arithmetic on many small values in a tight loop will pay for the provenance. Unwrap at the boundary and rewrap the result.
• When downstream code cannot accept the wrapper. Third-party libraries will coerce PValue to a plain float at the call boundary. Unwrap before calling, rewrap after.
• As a substitute for fixing the solver. The module makes the bug visible; it does not fix the solver. If a solver reports converged=True with a bogus residual of 0.0, the module trusts the solver's self-report.

Honest limitations

• Trusts the solver's self-report. A malicious or buggy solver can set converged=True with a residual of 0.0. The module has no way to verify the claim.
• Scalars only, plus PArray. No 2D matrix type. For matrix outputs, wrap each entry in a PArray of PArrays or use a separate PMatrix extension.
• Arithmetic covers + - * / **. No %, //, @ (matmul), or trigonometric functions. Add the ones you need by extending _arith.
• Comparisons strip provenance. pv < 5.0 returns a plain bool. If you need to propagate the fact that a comparison was made on unconverged data, use assert_converged first.
• cross_check runs solvers sequentially. No parallelism. For expensive solvers, run them yourself and pass the results to a manual Disagreement construction.
• jacobi_budget is calibrated for largest-element Jacobi. Other Jacobi variants (cyclic, threshold) have different scaling. The budget is an empirical formula, not a guarantee.
• No calibration against real pipelines. The module has not been instrumented inside a production numerical pipeline. The claim that it catches real bugs is demonstrated only on the synthetic demo.

The bug it prevents

The 97.705 from a real session would have looked like this in a provenance-aware pipeline:

value       : 97.705
converged   : False
iterations  : 200
residual    : 8.469e+00
tolerance   : 1.000e-10
repr        : PValue(97.705, UNCONVERGED, via='jacobi_short')
report(...) : <refused: 'jacobi_short' did not converge;
              residual=8.47, tol=1e-10>

The number never reaches a user. The number never reaches a paper. The failure is caught at the point of production, not the point of use.

Version history

version	change
0.1.0	Provenance, PValue, @solver, report(), assert_converged()
0.2.0	added PArray for sequences
0.3.0	added cross_check and Disagreement
0.4.0	jacobi_budget scales with matrix size
0.4.1	Disagreement.outlier() fixed: unconverged first, then loosest tolerance, then distance from mean

Reference

Part of a series of small tools built in one session:

tool	reads	answers
hv-manifold	a corpus	the geometry of style space
hv-reader	one text	how it reads
anomaly-or-bug	a number and a matrix	is this a bug or a discovery?
frontier-check	one claim	where does it sit relative to the frontier?
numerical-provenance	a numeric pipeline	can I trust this number?

The design principle — that a numeric value should carry its own convergence state rather than relying on the caller to track it — came out of a session in which an unconverged Jacobi silently returned 97.705 to a report. The tool was built to make that class of bug impossible.

License

Apache-2.0