src/docs/DETERMINISM.md

# Determinism: why the outputs are byte-identical, and where that stops

The engine reproduces the release's arithmetic, not just its results. Each piece below is pinned for a measured
reason; `docs/TESTING.md` lists the checks that hold it in place, and `tests/test_matrix.py` runs the same 39 files
under changing conditions (each GPU alone and all of them together, 1 to 64 threads, 1 to 64 files at once, other arrival orders,
a warm repeat, a fresh process with every BLAS thread variable at 64): every record and RTTM identical in all of them.

## The networks (GPU)

- **Same code and weights.** The release's own model classes (DiariZen and its pyannote fork at commit 844f555), the
  bundle's weights, built the way `load_bundle` builds them.
- **Same numeric flags, set after the imports.** FP32, TF32 off, deterministic algorithms, cuDNN benchmark off,
  `CUBLAS_WORKSPACE_CONFIG=:4096:8`. Importing DiariZen switches TF32 on for the whole process, so the flags are
  set after it, as `load_bundle` does, and re-asserted before every network call.
- **Same kernels.** torch 2.1.1+cu121, its cuDNN 8.9.2, and cuBLAS 12.1.0.26 (`env/canonical.sh` loads it from a
  CUDA 12.1 installation; the cu121 Triton image carries the same files and searches them first).
- **Same batch shapes.** Segmentation: batches of 8, the remainder, the zero-padded last window alone (other sizes
  change bits). Voiceprints: the trunk once per window in batches of 8, pooling and the last layer at the release's
  rows (identical bits), the odd last window as the release runs it.
- **One CUDA stream per GPU.** Every GPU of a kind gives the same bits as the others, but two streams sharing one
  model on one GPU changed about 1 batch in 1,000. Locally there is one worker thread per GPU; on Triton one process
  per GPU.
- **Models never cross GPUs.** Moving several model copies from GPU to GPU in one process gave all-NaN weights on
  one GPU; copies go through host memory, and every Triton instance sees only its own GPU.
- **Checked before use.** Every backend runs every batch shape at start-up and compares with the release's own
  outputs (`assets/nn_golden.json`, written from `load_bundle`): identical bytes where the setup is reproduced;
  otherwise the outputs must stay within fixed near-identical bounds of `assets/nn_reference.npz` (or, in strict
  mode, the backend stops).

## Everything else (CPU, Rust)

- **numpy, scipy, pyannote and CPython reproduced operation by operation** (crate `diar-exact`): pairwise
  summation with numpy's 8 accumulators and 8,192-element buffers, numpy's BLAS dispatch (gemm / gemv / dot) with
  numpy's own OpenBLAS binary at one thread, glibc's `exp`/`log`, introsort tie order, scipy's centroid linkage and
  heap, cosine distance with its two-lane sum, rectangular assignment, pyannote's sliding-window arithmetic and
  aggregation, CPython's `round`, `sum`, string sort and `.3f` formatting.
- **Parallel only where each element's arithmetic is unchanged.** Distances by row, VBx products by blocks of
  result rows (each its own single-threaded OpenBLAS call; tested bitwise against the one-call product), VBx and
  aggregation by rows or output frames with every element's terms in the release's order.
- **The engine's OpenBLAS is private** (its own linker namespace) and pinned to one thread whatever the environment
  says.
- **Decoding** is libsndfile 1.2.2, the binary soundfile loads, called in soundfile's order; non-16 kHz audio is
  resampled by the release's own torchaudio resampler (copied unchanged into `anuvani_diar.nn.resample`) with 2
  threads.

## Where the guarantee stops

- Identical bytes need the same software (`env/canonical.sh`) on the same kind of GPU as the release's outputs were
  produced with. Another GPU model, or another PyTorch, CUDA or cuDNN version, picks other kernels whose last bits
  differ: the start-up check then measures the networks against the release's outputs and runs only if they stay
  within the near-identical bounds (maximum posterior difference 0.25, frame decisions at least 99% equal, voiceprint
  cosine at least 0.995), and says so; strict mode refuses to run instead.
- The CPU side is pinned to AVX2 code paths by `env/canonical.sh` (numpy, ATen and oneDNN features, and
  `OPENBLAS_CORETYPE=Zen`). The engine's private OpenBLAS no longer depends on that variable: it sets
  `OPENBLAS_CORETYPE=Zen` (and one thread) itself for its own load when the CPU reports AVX2 and FMA, checks that it
  runs core Zen, and refuses to run otherwise (`docs/WHEEL.md`). On an AVX2 CPU the pins change nothing; on a CPU
  with AVX-512, running without them could select other kernels in numpy, ATen and oneDNN.
- Inputs the release itself would treat differently (a different bundle, batch size or package versions) are
  outside the contract.