| # The engine (crates/diar-engine), in plain words |
|
|
| The engine takes audio files and returns, for each, the RTTM text of the release's Indic model, byte-identical to |
| what the Python release writes (`release/anuvani_diar_v1/diarize.py --mode indic`). Only the two networks run in Python |
| (`python/anuvani_diar/nn/torch_backend.py`, the release's own model code and flags). Everything else is Rust. |
|
|
| ## The three pools |
|
|
| 1. **Input** (`decode_threads`, default 4). For each file, in input order: decode with the same libsndfile 1.2.2 |
| binary soundfile uses (`crates/diar-audio`), average the channels as numpy does, resample through the network |
| backend's copy of torchaudio's resampler when the rate is not 16 kHz, validate (empty or non-finite audio fails as in the |
| release), and plan the windows. The file's *blocks* then enter the device queue. |
| 2. **Devices** (one worker per backend *slot*; a slot is one GPU with one CUDA stream: two streams sharing a model on |
| one GPU were found to change about 1 batch in 1,000). |
| A block is 8 consecutive 16 s windows of one file (fewer at the end of a file). A worker runs, for its block: |
| - the release's segmentation batch(es): a full batch of 8, or the remainder batch and the zero-padded last |
| window alone, exactly as pyannote's `Inference.slide` partitions them; |
| - masks (argmax, 11-frame median, float16 record rounding) in Rust; |
| - the voiceprints: the ResNet trunk once per window in one batch of 8 (identical bits), pooling and |
| the last layer at the release's rows (2 windows x 4 speakers); an odd last window runs exactly as the |
| release runs it (batch of 4); |
| - and writes the three record arrays at the block's place. |
| Which block a free worker takes next is the *schedule*; it changes when a file finishes, never its bytes: |
| - `fifo` (the local engine's default): the earliest file first, its blocks in order. One long file spreads over |
| every GPU while later files wait. Best for batches. |
| - `fair` (the gateway's default): every file in progress advances one block per round, and a newly arrived file |
| starts at once, ahead of blocks further into earlier files. Short files finish quickly; long files absorb the |
| load while it lasts. |
| - `share`: processor sharing. Every file in progress advances one block per round, and a newly arrived file joins |
| from the next round (ties go to the older file), so no file is held back by newer ones, however many arrive. |
| 3. **Post-processing** (`post_threads`). When a file's last block lands, its record goes to this pool: |
| clustering (training selection, AHC dendrogram, PLDA features, VBx, cannot-link repair, assignment), |
| reconstruction, the decoder's features and trees, the RTTM text. The slow parts are parallel inside, always with |
| each element's arithmetic in the release's order: pairwise distances by row, VBx products by result rows (each |
| block its own single-threaded OpenBLAS call), VBx row-wise steps, and the aggregation by output frames. |
| |
| ## Bounds and failure |
|
|
| - At most `max_ready` files (default 2 x slots + 2) are admitted past the input stage at once; a file's decoded |
| audio is freed when its last block finishes and its place when its result is sent. |
| - A failure of one file (input, network, post-processing, or a panic anywhere in the engine) becomes that file's |
| error string, `"ExceptionName: message"` as the release would raise it; other files continue. |
| - Ctrl-C while iterating, or dropping the result iterator, cancels the run: no new file starts, the in-flight |
| network calls finish, the engine thread is joined. |
| - The engine's private OpenBLAS (numpy's own binary, loaded in a separate linker namespace) pins itself to one |
| thread, so no environment variable can change a bit. |
|
|
| ## What is exact, and why |
|
|
| | Stage | How identity is guaranteed | |
| |---|---| |
| | decode | same libsndfile binary, same call sequence as soundfile (tests: every format, 1,973 replay inputs) | |
| | channel mean, validation | numpy's float32 reduction reproduced; the release's checks and messages | |
| | windows and batches | pyannote's own slide / get_embeddings traced on CPU (26 lengths) | |
| | networks | the release's model objects and flags; the release's batch shapes (others change bits) | |
| | masks, float16 | release function on 300 seeded cases; float16 over all 2^32 inputs | |
| | post-processing | every cached record's RTTM (`make test-cache`); long files; per-stage differential tests | |
| | parallel arithmetic | each element keeps its sequential order; row-split dgemm tested bitwise (tests/test_npblas_par.py) | |
| |
