|
Download implementation-notes.md from SpeechAntiSpoofingBenchmarks/UTCN-XLSR2B-LogReg: direct link, hf CLI and curl.
- Browser
- Download file 7.37 kB
-
https://huggingface.co/SpeechAntiSpoofingBenchmarks/UTCN-XLSR2B-LogReg/resolve/main/implementation-notes.md
- Command line
-
hf download hf://SpeechAntiSpoofingBenchmarks/UTCN-XLSR2B-LogReg/implementation-notes.md
-
curl -L -o implementation-notes.md https://huggingface.co/SpeechAntiSpoofingBenchmarks/UTCN-XLSR2B-LogReg/resolve/main/implementation-notes.md
7.37 kB
| # UTCN-XLSR2B-LogReg — implementation notes | |
| Plan: `speech-spoof-bench/docs/plans/2026-11-09-thirteen-model-batch-arena-submission.md` (model #1). | |
| ## Source | |
| - HF `DavidCombei/deepfake-detection` @ `5f39744bc81762529e19f76e2bb7318e7e69f3fb`, audio core only | |
| (`benchmarks/_ckpt_dl/DavidCombei__deepfake-detection/audio-detection_core/`): `app.py`, `requirements.txt` | |
| (transformers 4.44, sklearn 1.3.2), `model/wav2vec2-xls-r-2b_truncated/{config.json,model.safetensors}` (182 fp32 | |
| tensors, 433,051,008 params, layers 0–8 only), `model/logreg_margin_pruning_ALL_with_scaler_threshold.joblib` | |
| (sha256 `dbdc1bd8…80dc`). No license anywhere on the repo → `LICENSE.source`; weights linked, not re-hosted. | |
| Both are symlinked into this dir. | |
| - Paper: Combei, Stan, Oneata, Müller, Cucu, "Unmasking real-world audio deepfakes: A data-centric approach", | |
| Interspeech 2025, arXiv 2506.09606. Training scripts: github.com/davidcombei/AI4T @ `2c3840cc` (CC-BY-4.0; read, not used). | |
| ## joblib → lr_head.npz (`export_lr_head.py`, log `logs_export_lr_head.txt`) | |
| - `pickletools` (no execution) shows protocol 4, globals `sklearn.linear_model._logistic.LogisticRegression`, | |
| `joblib.numpy_pickle.NumpyArrayWrapper`, `numpy.ndarray`, `numpy.dtype`; it stops at byte 528 because joblib writes | |
| raw array bytes inline after each wrapper. The full global set (found by the allow-list unpickler, which refuses | |
| anything else) adds only `sklearn.preprocessing._data.MinMaxScaler` (the scaler is MinMax, not Standard). | |
| - Loader: pure-Python `pickle._Unpickler` subclass; `find_class` allow-list maps the sklearn classes and the joblib | |
| wrapper to inert stubs (no sklearn/joblib code runs), `numpy.dtype` only for numeric codes; a custom BUILD handler | |
| reads each wrapper's array like joblib's non-object path (pad byte + 0xFF padding + nbytes, `np.frombuffer`). | |
| Never `joblib.load` / `pickle.load`. | |
| - Content: tuple `(LogisticRegression(C=1e6, lbfgs, classes [0,1], coef (1,1920), intercept −0.14833), | |
| MinMaxScaler(feature_range (0,1), clip False, fitted on 430,507 decision values, scale_ 0.0011718, min_ 0.56445), | |
| threshold 0.567134)`. Exported to `lr_head.npz` (sha256 `044bbb7e…6684`, deterministic). | |
| ## Wrapper (`utcn_xlsr2b_logreg.py:UTCNXLSR2BLogReg`, `_net.py`) | |
| - app.py path: `AutoFeatureExtractor(facebook/wav2vec2-xls-r-2b)` = per-utterance (x−mean)/sqrt(var+1e-7) | |
| (config vendored as `xlsr2b_preprocessor_config.json`, @3b6d89d0) → `CustomWav2Vec2Model` (48-layer build with | |
| `encoder.layers[:9]`) → `hidden_states[9]` → mean over time → `decision_function` → `MinMaxScaler.transform`; | |
| `pred = 1 ("real") iff value ≥ thresh − 5e-3` → class 1 = bona fide → **score = scaled decision value, higher = bona fide**. | |
| - Note: with only 9 layers, `hidden_states[9]` is the encoder's final entry = `encoder.layer_norm(layer-9 output)`. | |
| The authors' training extractor (AI4T `wav2vec2-xls-r-2b_all-layers_extractor.py`) used the full 48-layer model, | |
| where `hidden_states[9]` is the raw layer-9 output **without** that LayerNorm — so app.py's features differ from | |
| the training features. We replicate app.py (the released inference). Diagnostic on 300 random In-the-Wild clips | |
| (scratch, not shipped): EER 0.0% (whole clip) / 0.71% (4 s cap) for both variants, so app.py's path works. | |
| - Backbone: `Wav2Vec2Model(config, num_hidden_layers=9)` + `load_state_dict(safetensors, strict=True)`: all 182 | |
| tensors used, 0 missing / 0 unexpected (`masked_spec_embed` is loaded but unused in eval). | |
| - Attention: **eager**. transformers' default SDPA FP32 kernels are run-to-run non-deterministic here (scaled score | |
| jitter ~1e-4 per call, ~1e-3 vs eager); eager is bit-reproducible at the same speed (25.3 vs 25.3 utt/s on 5060). | |
| - Head: float64 numpy `X @ coef + intercept`, then `* scale + min` (exactly sklearn's ops; clip=False). | |
| - Input: Arena 4 s cap — first 64600 samples, shorter clips repeat-padded (np.tile), then normalised (app.py instead | |
| scores up to 60 s segments, skips segments < 0.7 s, averages sigmoid outputs). FP32, TF32 off. No resampling. | |
| - Params: 433,051,008 loaded; 433,049,088 forward (minus masked_spec_embed) + 1,921 LR → meta 433.051009. | |
| ## Checks | |
| - Tests (`test_utcn_xlsr2b_logreg.py`, 9 passed): crop/repeat-pad, normalisation == HF feature extractor, | |
| npz sha + head sign, strict load (182 tensors == module state), eval mode + param counts, batch == single | |
| (atol 1e-4; real batch of 4 with mixed lengths), determinism, bona fide > spoof on J-SPAW clips. | |
| - Source parity (`source_parity.py`, `logs_source_parity.txt`): app.py's model class + `from_pretrained` + HF feature | |
| extractor + real sklearn LogisticRegression/MinMaxScaler rebuilt from the npz, 18 J-SPAW_LA clips (16 + a 20000 and a | |
| 139944-sample clip). Source with eager attention: max |Δ| **0.0** (window off, whole clip) / **2.5e-8** (4 s cap). | |
| Source with its default SDPA: 1.24e-3 / 1.06e-3 (SDPA kernel noise, see above). | |
| - J-SPAW_LA full (PyTorch, RTX 5060 Ti idx 0, `run_pt.sh`): **EER 0.875%**, 2397 trials, 0 skipped. | |
| ## Batch sweep (`sweep.py`, 64600-sample input, 2026-11-09) | |
| | card | result | batch_size | | |
| |---|---|---| | |
| | RTX 4070 Ti SUPER (idx 2) | 107.3 utt/s @24, plateau 8–64 | 24 | | |
| | RTX 5060 Ti (idx 0) | flat 24.6–26.3 utt/s for bs 1–64 | 8 | | |
| ## TensorRT (`trt_utcn_xlsr2b_logreg.py`, sidecar `trt_utcn-xlsr2b-logreg.json`, TRT 10.16.1.11) | |
| Adapted from ProSDD-ASV19's script: the engine outputs the (B,1920) pooled features; the LR + MinMax head stays in | |
| numpy (the TRT class sets `self.head`). Strict FP32 (FORCE_FP32 and TF32 cleared by default); ONNX 1.65 GB < 2 GB, so | |
| the TorchScript exporter suffices. Build parity on 64 In-the-Wild clips. | |
| | card | precision | best batch | utt/s | build parity max Δ / Spearman | J-SPAW EER | vs PT Spearman / max Δ | | |
| |---|---|---|---|---|---|---| | |
| | RTX 4070 Ti SUPER (idx 2) | FP32 | 32 | 120.4 | 4.2e-7 / 1.0 | 0.875 (=PT) | 1.000000 / 1e-6 | | |
| | RTX 5060 Ti (idx 0) | FP32 | 32 | 72.5 | 2.3e-7 / 1.0 | 0.875 (=PT) | 1.000000 / 1e-6 | | |
| Both accepted (|ΔEER| = 0, Spearman ≥ 0.999). Only the 4070 ONNX is kept. Scripts: `run_trt_export.sh <gpu>`, | |
| `run_trt_jspaw.sh <gpu>`; compare with `cmp_scores.py`. | |
| ## Training data → trained_on | |
| Paper §2.1/§4.1: "ALL" = every sample (incl. test splits) of ASVspoof 2019 LA, FoR for-norm, ~60k random ASVspoof 2021 | |
| (Table 1: 22k bona fide / 589k spoof / 100 systems = the DF eval set), TIMIT-TTS, ODSS, MLAAD v5 + 80k M-AILABS bona | |
| fide, ASVspoof 5 train+dev; then margin pruning (§4.2). ITW and AI4T were held out of training, but the released | |
| configuration (ALL + margin-both pruning) is the one picked by In-the-Wild EER (1.70%, Table 4). | |
| - `trained_on`: ASVspoof2019_LA, ASVspoof2021_DF, ODSS, **InTheWild** (selection, following the Forensics-XLSR-Wild | |
| precedent — maintainer may drop it). DECRO is flagged automatically via its ASVspoof2019-LA:eval bona fide. | |
| - Not flagged: ASVspoof2021_LA (paper's ASV21 counts match DF), ASVspoof5 (train/dev only; Arena = eval), XMAD | |
| (M-AILABS bona fide is a separate corpus in provenance), FoR/TIMIT-TTS (not in the Arena). | |
| - Suggested `provenance.yaml` models entry (not added; maintainer): `utcn-xlsr2b-logreg: {backbone: XLS-R, | |
| trained_on: [ASVspoof2019-LA, "ASVspoof2021-DF:eval", "ASVspoof5:train", "ASVspoof5:dev", ODSS, MLAAD, M-AILABS, InTheWild]}`. | |