File size: 6,820 Bytes
6f93a12 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | # Pellav2 β implementation notes
Plan: `speech-spoof-bench/docs/plans/2026-11-09-thirteen-model-batch-arena-submission.md` (model #8).
## Source
- HF `Sadanie/pellav2-audio-deepfake-detector` @ `b668eb4dfe4719665692c73c1d70383680afff6b` (downloaded by
`_ckpt_dl/dl_batch4.py` into `_ckpt_dl/Sadanie__pellav2-audio-deepfake-detector/`). Card: `license: mit`, no
LICENSE file, no paper, training data not described. Files: `pellav2_detector.pt` (2.32 GB, sha256
`d21aedfb52e2f87706c2753715a29665482b0f4a034800c31579df5d458559ac`), `pellav2_infer.py` (sha256
`bf13cc97β¦6a67`), `config.json` (`{"name": "pellav2"}` only), README.
- **Architecture deviates from the plan row**: the plan says "XLS-R + learned weights over all 25 layers"; the
script is **w2v-BERT 2.0** (`facebook/w2v-bert-2.0`, 24 conformer layers, 580 M params) on 80-bin kaldi fbank
features (stacked to 160-d), softmax layer mix over the 25 hidden states, masked mean, `Linear(1024,1)`.
Layer mix + 25 layers as in the plan; only the backbone name differs.
- `pellav2_infer.py` read in full (it only imports torch/transformers/torchaudio/scipy/soundfile/pyloudnorm and
downloads from the Hub in `hub_weights()`); vendored into `_net.py`, never imported from the cache. A
byte-identical copy `_upstream_pellav2_infer.py` (sha256 asserted) is used only by `source_parity.py`.
- Backbone: the script calls `Wav2Vec2BertModel.from_pretrained("facebook/w2v-bert-2.0")` (unpinned) and then
overwrites every tensor with the .pt. We build `Wav2Vec2BertModel(config)` from that repo's `config.json` only
(pinned @ `da985ba0987f70aaeb84a80f2851cfac8c697a7b`, downloaded to `_ckpt_dl/facebook__w2v-bert-2.0/`, vendored
as `w2vbert2_config.json`); no backbone weights are downloaded.
- Weights: plain `OrderedDict` state_dict (776 fp32 tensors) β `torch.load(weights_only=True)` works without an
allow-list. `convert_ckpt.py` β `pellav2_detector.safetensors` (same keys/values, round-trip asserted; sha256
`050234557deabc95d9697a2ede42a5152ed06ba500a4488a42542ab793af428f`), written next to the download, symlinked here.
## Wrapper (`pellav2.py:Pellav2`, `_net.py`)
- Preprocessing = the script's `load_clip` after decoding (the runner hands 16 kHz mono float32, so the resample
branch is a no-op): mean removal β 2nd-order 20 Hz Butterworth high-pass (`sosfiltfilt`, whole clip, when
len > 480) β **centre 4 s (64000) crop** β gain to β23 LUFS (clipped Β±40 dB) β clip to [β1, 1] β zero-pad to
64000. The script's own window is already a 4 s crop, so it is kept as is (no Arena first-64600/repeat-pad
window; that would change both the crop position and the padding the fbank trim relies on).
- Features as the script: fbank on the trailing-zero-trimmed clip (β₯ 400 samples), Γ32768, per-utterance
mean/var norm, frame pairs stacked to 160-d, padded batch + attention mask (so a batch = one-at-a-time up to
float noise; the backbone masks attention and conformer convs).
- **pyloudnorm is not installed** in this environment (not in conda, no wheel in the pip/uv caches). Without it the
script silently falls back to an RMS-based level (its `except: pass`), so running the script here would NOT be
the authors' intended path. `_net.integrated_loudness` re-implements `pyloudnorm.Meter(16000).integrated_loudness`
(pyloudnorm 0.1.1: cookbook K-weighting high-shelf G=4 dB Q=1/β2 1500 Hz + high-pass Q=0.5 38 Hz via `lfilter`,
400 ms blocks with 75 % overlap, β70 LUFS absolute + β10 LU relative gating, same op order). Sanity: a 0 dBFS
997 Hz mono sine gives β3.083 LUFS at 16 kHz (BS.1770's 48 kHz reference β3.01; pyloudnorm's bilinear-warped
cookbook filters at 16 kHz account for the 0.07 dB), and with 48 kHz coefficients a β23 dBFS-RMS sine reads
β23.04. **Equality with the real package is not proven here**: `test_matches_pyloudnorm_if_installed` does it
(skipped now). To close it: `pip install pyloudnorm` (pure Python) and run the tests. A residual difference would
only be a per-clip gain offset of βͺ 1 dB.
- Score = βlogit(fake) (script: `p_fake = sigmoid(logit)`). FP32, TF32 off (cuDNN + matmul).
- Strict load: all 776 checkpoint tensors map 1:1 onto `Detector.state_dict()` (asserted set equality +
`strict=True`); `backbone.masked_spec_embed` (SpecAugment token) is loaded but unused at inference.
- Params: 580,494,170 total (all checkpoint tensors); 580,493,146 on the forward path (minus masked_spec_embed).
## Checks
- `test_pellav2.py`: crop/pad, RMS fallback, loudness target, BS.1770 reference tone, strict load, param count,
eval mode, batch == single (atol 1e-3), determinism, bona fide > spoof on J-SPAW. 9 passed, 1 skipped (pyloudnorm).
- `source_parity.py` (log `logs_source_parity.txt`): the authors' `Detector` + `load_clip` (from_pretrained swapped
for the pinned config, .pt via weights_only strict, pyloudnorm shimmed to `_net.integrated_loudness`, clips as
FLOAT wavs) one clip at a time vs the wrapper batched, on 16 J-SPAW_LA clips + 0.3 s (RMS-fallback branch) +
1.25 s + 8.6 s (centre-crop branch): preprocessing identical (max |Ξ| 0.0), **max |Ξscore| = 2.8e-6**.
The script has no full-utterance mode, so "window off" = "4 s cap" here.
## Batch sweep (`sweep.py`, 64000-sample view, 2026-11-10)
| card | best | notes |
|---|---|---|
| RTX 4070 Ti SUPER (idx 2) | 56.9 utt/s @24 | plateau 16β64 β `batch_size = 24` |
| RTX 5060 Ti (idx 1) | 25.1 utt/s @4 | ~18 at bs β₯ 8 β 4 on 5060 |
## Sanity (PyTorch, idx 2)
- Full J-SPAW_LA: **EER 0.000 %** (2397 trials, 0 skipped). Scores are saturated and fully separated (β β7.5 / +7.2).
This is not inverted and not random, but a perfect score points to J-SPAW (or the same source) in the undisclosed
training data. No trained_on entry, because the training data is unknown. Flagged for the maintainer.
## TensorRT (`trt_pellav2.py`, sidecar `trt_pellav2.json`, TRT 10.16.1.11)
The engine takes (feats[B,199,160], mask[B,199]). The fbank front-end stays in torch, padded to a fixed 199 frames.
The fp32 ONNX is 2.33 GB, so it uses the dynamo exporter with external data (`pellav2.onnx` + `.data`). FP16 build
parity on 64 InTheWild clips gave Spearman 0.99954 (5060) and 0.99977 (4070), below 0.9999, so both cards fall back
to **FP32** (TF32 cleared).
| card | precision | best batch | utt/s | build Spearman / max Ξ | J-SPAW EER | J-SPAW Spearman vs PT / max Ξ |
|---|---|---|---|---|---|---|
| RTX 4070 Ti SUPER (idx 2) | FP32 | 12 | 68.4 | 1.0 / 7.6e-6 | 0.000 (= PT) | 1.000000 / 1.0e-5 |
| RTX 5060 Ti (idx 1) | FP32 | 16 | 49.7 | 1.0 / 2.9e-6 | 0.000 (= PT) | 1.000000 / 1.4e-5 |
Both cards are accepted. Scripts: `run_pt.sh`, `run_trt_export.sh <2|1>` (run idx 2 first, it writes the ONNX),
`run_trt_jspaw.sh <2|1>`, `cmp_scores.py`.
|