Download implementation-notes.md from SpeechAntiSpoofingBenchmarks/Pellav3: direct link, hf CLI and curl.
- Browser
- Download file 8.29 kB
-
https://huggingface.co/SpeechAntiSpoofingBenchmarks/Pellav3/resolve/main/implementation-notes.md
- Command line
-
hf download hf://SpeechAntiSpoofingBenchmarks/Pellav3/implementation-notes.md
-
curl -L -o implementation-notes.md https://huggingface.co/SpeechAntiSpoofingBenchmarks/Pellav3/resolve/main/implementation-notes.md
Pellav3 — implementation notes
Plan: speech-spoof-bench/docs/plans/2026-11-09-thirteen-model-batch-arena-submission.md (model #9).
Source
- HF
Sadanie/pellav3-audio-deepfake-detector@47b69420803c01de65af264ecdc2c5f35f2184f3(_ckpt_dl/dl_batch4.py→_ckpt_dl/Sadanie__pellav3-audio-deepfake-detector/). Card:license: mit, no LICENSE file, no paper, training data not described, "1.54 billion parameters, 16 kHz input". Files:pellav3_detector.pt(6.18 GB fp32, sha256ac6ae020ac5de06beee4f283ad1ea1fb1ae88dd8da19f483a602a032966695ec),pellav3_infer.py(sha256c38f9a43…ae6e),config.json({"name": …}only), README. - Net (script): two views of each 4 s window — XLS-R 1B (48 layers) on the per-window standardised waveform with a softmax mix of its 49 hidden states → Linear(1280,256) + 2 Res2 blocks; w2v-BERT 2.0 on 80-bin kaldi fbank (stacked 160-d, masked) with a 25-state mix → Linear(1024,256) + 2 Res2 blocks; frames concatenated (512) → 2 shared Res2 blocks → attentive statistics pooling → Linear(1024,1) = logit(fake).
pellav3_infer.pyread in full (subprocess only forffmpegdecoding, Hub only inhub_weights()); vendored into_net.py. Byte-identical copy_upstream_pellav3_infer.py(sha256 asserted) is used only bysource_parity.py.- Backbones: the script itself builds them from configs (
Model(Config.from_pretrained(id)), random init) and fills them from the .pt; we use the same configs, pinned and vendored:xlsr1b_config.json= facebook/wav2vec2-xls-r-1b @35eaea9a0ed0f97592277d79208e40ab8917d1e3,w2vbert2_config.json= facebook/w2v-bert-2.0 @da985ba0987f70aaeb84a80f2851cfac8c697a7b(only config.json downloaded, to_ckpt_dl/facebook__*/). - Weights: plain state_dict (1647 fp32 tensors, pos_conv already in
parametrizations.weight.original0/1form) →torch.load(weights_only=True)without allow-list.convert_ckpt.py→pellav3_detector.safetensors(same keys/values, asserted; sha2566a721d66a0b9e1a1011193a66f245a4836dbfc036eb671cd1ff80d6e266d5a73), symlinked here.
transformers version (script pins 4.48.3; we run 5.3.0)
4.48.3 is not available offline; the uv cache has 4.49.0. tf_version_check.py builds both backbones with the
Pellav3 weights under 4.49.0 (dependency check stubbed) and under 5.3.0 and compares all hidden states on fixed inputs
(XLS-R on a waveform batch with a zero-padded item; w2v-BERT on features with a padded attention mask), CPU FP32:
max |Δ| = 0.0 for both (logs_tf_version_check.txt). 4.48.3 → 4.49.0 was not tested directly (no 4.48 wheel
offline); the wav2vec2 / wav2vec2_bert eval paths had no numerical changes in that patch range that we know of.
Wrapper (pellav3.py:Pellav3, _net.py)
- Window: the script scores clips ≤ 4 s once and longer clips with 4 s windows every 2 s (last window ending at the
clip end), averaging window logits. Arena 4 s cap: the wrapper scores one window, the grid's first one
(
decode(path, 0, 4.0)==x[:64000]) through the script'sview()(mean removal, 20 Hz 2nd-order Butterworth high-pass bysosfiltfilton the window, centre crop (no-op), gain to −23 LUFS clipped ±40 dB, clip [−1, 1], zero-pad to 64000). For clips ≤ 4 s (all of J-SPAW_LA) this is exactly the authors' score. Zero-padding (not repeat) is kept because the w2v-BERT branch trims trailing zeros before its fbank, as the authors trained it. - Loudness: pyloudnorm is not installed here (see Pellav2 notes for details); the same re-implementation of
pyloudnorm.Meter(16000).integrated_loudnessis used, with the script's RMS fallback verbatim. Equality with the real package is untested (test_matches_pyloudnorm_if_installedis skipped). - Batch invariance (edit 6): in the script's padded batch, F = min(XLS-R frames, batch-max w2v-BERT frames), and
the XLS-R frames beyond an item's own w2v-BERT length are not zeroed, so the shared Res2 blocks' dilated convs read
them at the mask boundary — a clip's logit depended on its batch-mates (|Δ| up to 7.7e-3 vs one-at-a-time in our
smoke test).
Detector(batch_invariant=True)zeros those frames; for B = 1 it is a no-op (asserted in tests), so the wrapper equals the authors' one-clip result in any batch (|Δ| 7.6e-6). - Precision: the script runs the net under
torch.autocast(bfloat16)on CUDA. The wrapper is FP32 (TF32 off), per the batch instruction. Measured: the script's own bf16score()vs our FP32 on the same single window (clips ≤ 4 s) differs by ≤ 0.18 logit (logits ~±20; bf16 resolution there is 0.125). - Score = −logit(fake). Strict load: all 1647 tensors map 1:1 onto
Detector.state_dict()(set equality asserted,strict=True). Loaded but unused at inference (the script keeps them "for the weights file"), asserted allow-listUNUSED_AT_INFERENCE:xl.backbone.masked_spec_embed,w2v.backbone.masked_spec_embed(SpecAugment tokens),xl.head.*,w2v.head.*,w2v.att.*(per-branch auxiliary heads/pooling): 37,123 params. - Params: 1,544,130,575 total; 1,544,093,452 on the forward path.
Checks
test_pellav3.py: first-window/view/window-grid, short-clip pad + RMS fallback, loudness target, reference tone, strict load + allow-list size, param count, eval mode, batch == single, batch_invariant no-op for B = 1, determinism, bona fide > spoof on J-SPAW. 11 passed, 1 skipped (pyloudnorm).source_parity.py(logs_source_parity.txt), authors' Detector + decode (ffmpeg) + view/views/score on FLOAT wavs of 16 J-SPAW_LA clips + 0.3 s + 1.25 s + 6.6 s (3 windows) + 8.6 s (4 windows):- preprocessing identical (cap and all windows: max |Δ| 0.0);
- 4 s cap (wrapper batched FP32 vs script net on
view(decode(p, 0, 4)), one clip, FP32): max |Δ| 7.6e-6; - window off (wrapper
windows()+ mean vs scriptviews()+ mean, FP32): max |Δ| 0.0; - script's own bf16
score()vs wrapper FP32 cap: 1.23 overall (long clips: different windows), 0.18 on ≤ 4 s clips.
Batch sweep (sweep.py, 64000-sample view, 2026-11-10)
| card | best | notes |
|---|---|---|
| RTX 4070 Ti SUPER (idx 2) | 26.3 utt/s @24 | plateau 8–32 → batch_size = 24 |
| RTX 5060 Ti (idx 1) | 8.35 utt/s @2 | ~7 at bs ≥ 4 → 2 on 5060 |
Sanity (PyTorch, idx 2)
- Full J-SPAW_LA: EER 0.000 % (2397 trials, 0 skipped). Logits are saturated and fully separated (about ±20), which suggests possible training-data overlap (undisclosed). Flagged for the maintainer.
TensorRT (trt_pellav3.py, sidecar trt_pellav3.json)
The engine takes (wav[B,64000], feats[B,199,160], mask[B,199]); the fbank stays in torch. The ONNX is 5.9 GB, so it
uses the dynamo exporter with external data. On RTX 4070 Ti SUPER, FP16 build Spearman was 0.99945, below 0.9999, so
it fell back to FP32: build parity Spearman 1.0, max Δ 3.4e-4, best batch 8, 31.7 utt/s.
Outcome (maintainer, 2026-11-10): PyTorch only. The full J-SPAW_LA run with Pellav3TRT skipped all 2397 clips
('Detector' object has no attribute 'w2v': the TRT class drops a submodule the scoring path still uses), and the
engine is only ~20% faster than PyTorch on the 4070 (31.7 vs 26.3 utt/s), so the class was not fixed and the 5060
engine build was stopped. All drivers (lanes, EmoSpoofTTS, LRLspoof) use pellav3:Pellav3.
Update 2026-11-11: TensorRT on RTX 5060 Ti (adopted for lanes)
The _features fix made Pellav3TRT score correctly (4070 engine vs PyTorch, 2072 J-SPAW clips: Spearman 1.0, max Δ 1.9e-4).
The 4070 was then removed from the machine, and on the 5060 Ti PyTorch runs at only 8.4 utt/s. So a FP32 engine was built
for RTX 5060 Ti (2.4x).
Build parity on 64 clips: Spearman 0.99918, max Δ 0.41 (FP32 engine, TF32 tactics). Full-dataset parity vs PyTorch:
J-SPAW_LA n=2397 Spearman 1.000000, max Δ 2.1e-4, EER 0.000 = 0.000; DeepVoice n=5053 Spearman 1.000000,
max Δ 2.6e-4, EER 1.9108 = 1.9108 (identical). Adopted for the remaining lane datasets on 5060 cards (4060 Ti has no
engine and stays PyTorch). Cells already submitted were computed in PyTorch; J-SPAW_LA stays submitted from PyTorch.TRT_FORCE_FP32=1; the FP16 build parity had failed on the 4070): 20.2 utt/s at batch 8 (