# Pellav3 — implementation notes Plan: `speech-spoof-bench/docs/plans/2026-11-09-thirteen-model-batch-arena-submission.md` (model #9). ## Source - HF `Sadanie/pellav3-audio-deepfake-detector` @ `47b69420803c01de65af264ecdc2c5f35f2184f3` (`_ckpt_dl/dl_batch4.py` → `_ckpt_dl/Sadanie__pellav3-audio-deepfake-detector/`). Card: `license: mit`, no LICENSE file, no paper, training data not described, "1.54 billion parameters, 16 kHz input". Files: `pellav3_detector.pt` (6.18 GB fp32, sha256 `ac6ae020ac5de06beee4f283ad1ea1fb1ae88dd8da19f483a602a032966695ec`), `pellav3_infer.py` (sha256 `c38f9a43…ae6e`), `config.json` (`{"name": …}` only), README. - Net (script): two views of each 4 s window — **XLS-R 1B** (48 layers) on the per-window standardised waveform with a softmax mix of its 49 hidden states → Linear(1280,256) + 2 Res2 blocks; **w2v-BERT 2.0** on 80-bin kaldi fbank (stacked 160-d, masked) with a 25-state mix → Linear(1024,256) + 2 Res2 blocks; frames concatenated (512) → 2 shared Res2 blocks → attentive statistics pooling → Linear(1024,1) = logit(fake). - `pellav3_infer.py` read in full (subprocess only for `ffmpeg` decoding, Hub only in `hub_weights()`); vendored into `_net.py`. Byte-identical copy `_upstream_pellav3_infer.py` (sha256 asserted) is used only by `source_parity.py`. - Backbones: the script itself builds them from configs (`Model(Config.from_pretrained(id))`, random init) and fills them from the .pt; we use the same configs, pinned and vendored: `xlsr1b_config.json` = facebook/wav2vec2-xls-r-1b @ `35eaea9a0ed0f97592277d79208e40ab8917d1e3`, `w2vbert2_config.json` = facebook/w2v-bert-2.0 @ `da985ba0987f70aaeb84a80f2851cfac8c697a7b` (only config.json downloaded, to `_ckpt_dl/facebook__*/`). - Weights: plain state_dict (1647 fp32 tensors, pos_conv already in `parametrizations.weight.original0/1` form) → `torch.load(weights_only=True)` without allow-list. `convert_ckpt.py` → `pellav3_detector.safetensors` (same keys/values, asserted; sha256 `6a721d66a0b9e1a1011193a66f245a4836dbfc036eb671cd1ff80d6e266d5a73`), symlinked here. ## transformers version (script pins 4.48.3; we run 5.3.0) 4.48.3 is not available offline; the uv cache has **4.49.0**. `tf_version_check.py` builds both backbones with the Pellav3 weights under 4.49.0 (dependency check stubbed) and under 5.3.0 and compares all hidden states on fixed inputs (XLS-R on a waveform batch with a zero-padded item; w2v-BERT on features with a padded attention mask), CPU FP32: **max |Δ| = 0.0 for both** (`logs_tf_version_check.txt`). 4.48.3 → 4.49.0 was not tested directly (no 4.48 wheel offline); the wav2vec2 / wav2vec2_bert eval paths had no numerical changes in that patch range that we know of. ## Wrapper (`pellav3.py:Pellav3`, `_net.py`) - **Window**: the script scores clips ≤ 4 s once and longer clips with 4 s windows every 2 s (last window ending at the clip end), averaging window logits. Arena 4 s cap: the wrapper scores **one window, the grid's first one** (`decode(path, 0, 4.0)` == `x[:64000]`) through the script's `view()` (mean removal, 20 Hz 2nd-order Butterworth high-pass by `sosfiltfilt` on the window, centre crop (no-op), gain to −23 LUFS clipped ±40 dB, clip [−1, 1], zero-pad to 64000). For clips ≤ 4 s (all of J-SPAW_LA) this is exactly the authors' score. Zero-padding (not repeat) is kept because the w2v-BERT branch trims trailing zeros before its fbank, as the authors trained it. - **Loudness**: pyloudnorm is not installed here (see Pellav2 notes for details); the same re-implementation of `pyloudnorm.Meter(16000).integrated_loudness` is used, with the script's RMS fallback verbatim. Equality with the real package is untested (`test_matches_pyloudnorm_if_installed` is skipped). - **Batch invariance (edit 6)**: in the script's padded batch, F = min(XLS-R frames, *batch-max* w2v-BERT frames), and the XLS-R frames beyond an item's own w2v-BERT length are not zeroed, so the shared Res2 blocks' dilated convs read them at the mask boundary — a clip's logit depended on its batch-mates (|Δ| up to 7.7e-3 vs one-at-a-time in our smoke test). `Detector(batch_invariant=True)` zeros those frames; for B = 1 it is a no-op (asserted in tests), so the wrapper equals the authors' one-clip result in any batch (|Δ| 7.6e-6). - **Precision**: the script runs the net under `torch.autocast(bfloat16)` on CUDA. The wrapper is FP32 (TF32 off), per the batch instruction. Measured: the script's own bf16 `score()` vs our FP32 on the same single window (clips ≤ 4 s) differs by ≤ 0.18 logit (logits ~±20; bf16 resolution there is 0.125). - Score = −logit(fake). Strict load: all 1647 tensors map 1:1 onto `Detector.state_dict()` (set equality asserted, `strict=True`). Loaded but unused at inference (the script keeps them "for the weights file"), asserted allow-list `UNUSED_AT_INFERENCE`: `xl.backbone.masked_spec_embed`, `w2v.backbone.masked_spec_embed` (SpecAugment tokens), `xl.head.*`, `w2v.head.*`, `w2v.att.*` (per-branch auxiliary heads/pooling): 37,123 params. - Params: 1,544,130,575 total; 1,544,093,452 on the forward path. ## Checks - `test_pellav3.py`: first-window/view/window-grid, short-clip pad + RMS fallback, loudness target, reference tone, strict load + allow-list size, param count, eval mode, batch == single, batch_invariant no-op for B = 1, determinism, bona fide > spoof on J-SPAW. 11 passed, 1 skipped (pyloudnorm). - `source_parity.py` (`logs_source_parity.txt`), authors' Detector + decode (ffmpeg) + view/views/score on FLOAT wavs of 16 J-SPAW_LA clips + 0.3 s + 1.25 s + 6.6 s (3 windows) + 8.6 s (4 windows): - preprocessing identical (cap and all windows: max |Δ| 0.0); - **4 s cap** (wrapper batched FP32 vs script net on `view(decode(p, 0, 4))`, one clip, FP32): max |Δ| 7.6e-6; - **window off** (wrapper `windows()` + mean vs script `views()` + mean, FP32): max |Δ| 0.0; - script's own bf16 `score()` vs wrapper FP32 cap: 1.23 overall (long clips: different windows), 0.18 on ≤ 4 s clips. ## Batch sweep (`sweep.py`, 64000-sample view, 2026-11-10) | card | best | notes | |---|---|---| | RTX 4070 Ti SUPER (idx 2) | 26.3 utt/s @24 | plateau 8–32 → `batch_size = 24` | | RTX 5060 Ti (idx 1) | 8.35 utt/s @2 | ~7 at bs ≥ 4 → 2 on 5060 | ## Sanity (PyTorch, idx 2) - Full J-SPAW_LA: **EER 0.000 %** (2397 trials, 0 skipped). Logits are saturated and fully separated (about ±20), which suggests possible training-data overlap (undisclosed). Flagged for the maintainer. ## TensorRT (`trt_pellav3.py`, sidecar `trt_pellav3.json`) The engine takes (wav[B,64000], feats[B,199,160], mask[B,199]); the fbank stays in torch. The ONNX is 5.9 GB, so it uses the dynamo exporter with external data. On RTX 4070 Ti SUPER, FP16 build Spearman was 0.99945, below 0.9999, so it fell back to **FP32**: build parity Spearman 1.0, max Δ 3.4e-4, best batch 8, 31.7 utt/s. Outcome (maintainer, 2026-11-10): **PyTorch only.** The full J-SPAW_LA run with `Pellav3TRT` skipped all 2397 clips (`'Detector' object has no attribute 'w2v'`: the TRT class drops a submodule the scoring path still uses), and the engine is only ~20% faster than PyTorch on the 4070 (31.7 vs 26.3 utt/s), so the class was not fixed and the 5060 engine build was stopped. All drivers (lanes, EmoSpoofTTS, LRLspoof) use `pellav3:Pellav3`. ### Update 2026-11-11: TensorRT on RTX 5060 Ti (adopted for lanes) The `_features` fix made `Pellav3TRT` score correctly (4070 engine vs PyTorch, 2072 J-SPAW clips: Spearman 1.0, max Δ 1.9e-4). The 4070 was then removed from the machine, and on the 5060 Ti PyTorch runs at only ~8.4 utt/s. So a FP32 engine was built for RTX 5060 Ti (`TRT_FORCE_FP32=1`; the FP16 build parity had failed on the 4070): 20.2 utt/s at batch 8 (~2.4x). Build parity on 64 clips: Spearman 0.99918, max Δ 0.41 (FP32 engine, TF32 tactics). Full-dataset parity vs PyTorch: J-SPAW_LA n=2397 Spearman 1.000000, max Δ 2.1e-4, EER 0.000 = 0.000; DeepVoice n=5053 Spearman 1.000000, max Δ 2.6e-4, EER 1.9108 = 1.9108 (identical). Adopted for the remaining lane datasets on 5060 cards (4060 Ti has no engine and stays PyTorch). Cells already submitted were computed in PyTorch; J-SPAW_LA stays submitted from PyTorch.