File size: 6,820 Bytes
6f93a12
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
# Pellav2 β€” implementation notes

Plan: `speech-spoof-bench/docs/plans/2026-11-09-thirteen-model-batch-arena-submission.md` (model #8).

## Source
- HF `Sadanie/pellav2-audio-deepfake-detector` @ `b668eb4dfe4719665692c73c1d70383680afff6b` (downloaded by
  `_ckpt_dl/dl_batch4.py` into `_ckpt_dl/Sadanie__pellav2-audio-deepfake-detector/`). Card: `license: mit`, no
  LICENSE file, no paper, training data not described. Files: `pellav2_detector.pt` (2.32 GB, sha256
  `d21aedfb52e2f87706c2753715a29665482b0f4a034800c31579df5d458559ac`), `pellav2_infer.py` (sha256
  `bf13cc97…6a67`), `config.json` (`{"name": "pellav2"}` only), README.
- **Architecture deviates from the plan row**: the plan says "XLS-R + learned weights over all 25 layers"; the
  script is **w2v-BERT 2.0** (`facebook/w2v-bert-2.0`, 24 conformer layers, 580 M params) on 80-bin kaldi fbank
  features (stacked to 160-d), softmax layer mix over the 25 hidden states, masked mean, `Linear(1024,1)`.
  Layer mix + 25 layers as in the plan; only the backbone name differs.
- `pellav2_infer.py` read in full (it only imports torch/transformers/torchaudio/scipy/soundfile/pyloudnorm and
  downloads from the Hub in `hub_weights()`); vendored into `_net.py`, never imported from the cache. A
  byte-identical copy `_upstream_pellav2_infer.py` (sha256 asserted) is used only by `source_parity.py`.
- Backbone: the script calls `Wav2Vec2BertModel.from_pretrained("facebook/w2v-bert-2.0")` (unpinned) and then
  overwrites every tensor with the .pt. We build `Wav2Vec2BertModel(config)` from that repo's `config.json` only
  (pinned @ `da985ba0987f70aaeb84a80f2851cfac8c697a7b`, downloaded to `_ckpt_dl/facebook__w2v-bert-2.0/`, vendored
  as `w2vbert2_config.json`); no backbone weights are downloaded.
- Weights: plain `OrderedDict` state_dict (776 fp32 tensors) β†’ `torch.load(weights_only=True)` works without an
  allow-list. `convert_ckpt.py` β†’ `pellav2_detector.safetensors` (same keys/values, round-trip asserted; sha256
  `050234557deabc95d9697a2ede42a5152ed06ba500a4488a42542ab793af428f`), written next to the download, symlinked here.

## Wrapper (`pellav2.py:Pellav2`, `_net.py`)
- Preprocessing = the script's `load_clip` after decoding (the runner hands 16 kHz mono float32, so the resample
  branch is a no-op): mean removal β†’ 2nd-order 20 Hz Butterworth high-pass (`sosfiltfilt`, whole clip, when
  len > 480) β†’ **centre 4 s (64000) crop** β†’ gain to βˆ’23 LUFS (clipped Β±40 dB) β†’ clip to [βˆ’1, 1] β†’ zero-pad to
  64000. The script's own window is already a 4 s crop, so it is kept as is (no Arena first-64600/repeat-pad
  window; that would change both the crop position and the padding the fbank trim relies on).
- Features as the script: fbank on the trailing-zero-trimmed clip (β‰₯ 400 samples), Γ—32768, per-utterance
  mean/var norm, frame pairs stacked to 160-d, padded batch + attention mask (so a batch = one-at-a-time up to
  float noise; the backbone masks attention and conformer convs).
- **pyloudnorm is not installed** in this environment (not in conda, no wheel in the pip/uv caches). Without it the
  script silently falls back to an RMS-based level (its `except: pass`), so running the script here would NOT be
  the authors' intended path. `_net.integrated_loudness` re-implements `pyloudnorm.Meter(16000).integrated_loudness`
  (pyloudnorm 0.1.1: cookbook K-weighting high-shelf G=4 dB Q=1/√2 1500 Hz + high-pass Q=0.5 38 Hz via `lfilter`,
  400 ms blocks with 75 % overlap, βˆ’70 LUFS absolute + βˆ’10 LU relative gating, same op order). Sanity: a 0 dBFS
  997 Hz mono sine gives βˆ’3.083 LUFS at 16 kHz (BS.1770's 48 kHz reference βˆ’3.01; pyloudnorm's bilinear-warped
  cookbook filters at 16 kHz account for the 0.07 dB), and with 48 kHz coefficients a βˆ’23 dBFS-RMS sine reads
  βˆ’23.04. **Equality with the real package is not proven here**: `test_matches_pyloudnorm_if_installed` does it
  (skipped now). To close it: `pip install pyloudnorm` (pure Python) and run the tests. A residual difference would
  only be a per-clip gain offset of β‰ͺ 1 dB.
- Score = βˆ’logit(fake) (script: `p_fake = sigmoid(logit)`). FP32, TF32 off (cuDNN + matmul).
- Strict load: all 776 checkpoint tensors map 1:1 onto `Detector.state_dict()` (asserted set equality +
  `strict=True`); `backbone.masked_spec_embed` (SpecAugment token) is loaded but unused at inference.
- Params: 580,494,170 total (all checkpoint tensors); 580,493,146 on the forward path (minus masked_spec_embed).

## Checks
- `test_pellav2.py`: crop/pad, RMS fallback, loudness target, BS.1770 reference tone, strict load, param count,
  eval mode, batch == single (atol 1e-3), determinism, bona fide > spoof on J-SPAW. 9 passed, 1 skipped (pyloudnorm).
- `source_parity.py` (log `logs_source_parity.txt`): the authors' `Detector` + `load_clip` (from_pretrained swapped
  for the pinned config, .pt via weights_only strict, pyloudnorm shimmed to `_net.integrated_loudness`, clips as
  FLOAT wavs) one clip at a time vs the wrapper batched, on 16 J-SPAW_LA clips + 0.3 s (RMS-fallback branch) +
  1.25 s + 8.6 s (centre-crop branch): preprocessing identical (max |Ξ”| 0.0), **max |Ξ”score| = 2.8e-6**.
  The script has no full-utterance mode, so "window off" = "4 s cap" here.

## Batch sweep (`sweep.py`, 64000-sample view, 2026-11-10)
| card | best | notes |
|---|---|---|
| RTX 4070 Ti SUPER (idx 2) | 56.9 utt/s @24 | plateau 16–64 β†’ `batch_size = 24` |
| RTX 5060 Ti (idx 1) | 25.1 utt/s @4 | ~18 at bs β‰₯ 8 β†’ 4 on 5060 |

## Sanity (PyTorch, idx 2)
- Full J-SPAW_LA: **EER 0.000 %** (2397 trials, 0 skipped). Scores are saturated and fully separated (β‰ˆ βˆ’7.5 / +7.2).
  This is not inverted and not random, but a perfect score points to J-SPAW (or the same source) in the undisclosed
  training data. No trained_on entry, because the training data is unknown. Flagged for the maintainer.

## TensorRT (`trt_pellav2.py`, sidecar `trt_pellav2.json`, TRT 10.16.1.11)
The engine takes (feats[B,199,160], mask[B,199]). The fbank front-end stays in torch, padded to a fixed 199 frames.
The fp32 ONNX is 2.33 GB, so it uses the dynamo exporter with external data (`pellav2.onnx` + `.data`). FP16 build
parity on 64 InTheWild clips gave Spearman 0.99954 (5060) and 0.99977 (4070), below 0.9999, so both cards fall back
to **FP32** (TF32 cleared).
| card | precision | best batch | utt/s | build Spearman / max Ξ” | J-SPAW EER | J-SPAW Spearman vs PT / max Ξ” |
|---|---|---|---|---|---|---|
| RTX 4070 Ti SUPER (idx 2) | FP32 | 12 | 68.4 | 1.0 / 7.6e-6 | 0.000 (= PT) | 1.000000 / 1.0e-5 |
| RTX 5060 Ti (idx 1) | FP32 | 16 | 49.7 | 1.0 / 2.9e-6 | 0.000 (= PT) | 1.000000 / 1.4e-5 |
Both cards are accepted. Scripts: `run_pt.sh`, `run_trt_export.sh <2|1>` (run idx 2 first, it writes the ONNX),
`run_trt_jspaw.sh <2|1>`, `cmp_scores.py`.