src/docs/ENGINE.md

# The engine (crates/diar-engine), in plain words

The engine takes audio files and returns, for each, the RTTM text of the release's Indic model, byte-identical to
what the Python release writes (`release/anuvani_diar_v1/diarize.py --mode indic`). Only the two networks run in Python
(`python/anuvani_diar/nn/torch_backend.py`, the release's own model code and flags). Everything else is Rust.

## The three pools

1. **Input** (`decode_threads`, default 4). For each file, in input order: decode with the same libsndfile 1.2.2
   binary soundfile uses (`crates/diar-audio`), average the channels as numpy does, resample through the network
   backend's copy of torchaudio's resampler when the rate is not 16 kHz, validate (empty or non-finite audio fails as in the
   release), and plan the windows. The file's *blocks* then enter the device queue.
2. **Devices** (one worker per backend *slot*; a slot is one GPU with one CUDA stream: two streams sharing a model on
   one GPU were found to change about 1 batch in 1,000).
   A block is 8 consecutive 16 s windows of one file (fewer at the end of a file). A worker runs, for its block:
   - the release's segmentation batch(es): a full batch of 8, or the remainder batch and the zero-padded last
     window alone, exactly as pyannote's `Inference.slide` partitions them;
   - masks (argmax, 11-frame median, float16 record rounding) in Rust;
   - the voiceprints: the ResNet trunk once per window in one batch of 8 (identical bits), pooling and
     the last layer at the release's rows (2 windows x 4 speakers); an odd last window runs exactly as the
     release runs it (batch of 4);
   - and writes the three record arrays at the block's place.
   Which block a free worker takes next is the *schedule*; it changes when a file finishes, never its bytes:
   - `fifo` (the local engine's default): the earliest file first, its blocks in order. One long file spreads over
     every GPU while later files wait. Best for batches.
   - `fair` (the gateway's default): every file in progress advances one block per round, and a newly arrived file
     starts at once, ahead of blocks further into earlier files. Short files finish quickly; long files absorb the
     load while it lasts.
   - `share`: processor sharing. Every file in progress advances one block per round, and a newly arrived file joins
     from the next round (ties go to the older file), so no file is held back by newer ones, however many arrive.
3. **Post-processing** (`post_threads`). When a file's last block lands, its record goes to this pool:
   clustering (training selection, AHC dendrogram, PLDA features, VBx, cannot-link repair, assignment),
   reconstruction, the decoder's features and trees, the RTTM text. The slow parts are parallel inside, always with
   each element's arithmetic in the release's order: pairwise distances by row, VBx products by result rows (each
   block its own single-threaded OpenBLAS call), VBx row-wise steps, and the aggregation by output frames.

## Bounds and failure

- At most `max_ready` files (default 2 x slots + 2) are admitted past the input stage at once; a file's decoded
  audio is freed when its last block finishes and its place when its result is sent.
- A failure of one file (input, network, post-processing, or a panic anywhere in the engine) becomes that file's
  error string, `"ExceptionName: message"` as the release would raise it; other files continue.
- Ctrl-C while iterating, or dropping the result iterator, cancels the run: no new file starts, the in-flight
  network calls finish, the engine thread is joined.
- The engine's private OpenBLAS (numpy's own binary, loaded in a separate linker namespace) pins itself to one
  thread, so no environment variable can change a bit.

## What is exact, and why

| Stage | How identity is guaranteed |
|---|---|
| decode | same libsndfile binary, same call sequence as soundfile (tests: every format, 1,973 replay inputs) |
| channel mean, validation | numpy's float32 reduction reproduced; the release's checks and messages |
| windows and batches | pyannote's own slide / get_embeddings traced on CPU (26 lengths) |
| networks | the release's model objects and flags; the release's batch shapes (others change bits) |
| masks, float16 | release function on 300 seeded cases; float16 over all 2^32 inputs |
| post-processing | every cached record's RTTM (`make test-cache`); long files; per-stage differential tests |
| parallel arithmetic | each element keeps its sequential order; row-split dgemm tested bitwise (tests/test_npblas_par.py) |