Hugging Face's logo

SpeechAntiSpoofingBenchmarks
/
DeepFense-CodecFake-W2V2-MLP

PyTorch
audio
anti-spoofing
deepfake-detection
speech-anti-spoofing-arena
Model card Files
xet
 Community
DeepFense-CodecFake-W2V2-MLP / implementation-notes.md
korallll's picture
korallll
Wrapper code (speech-spoof-bench)
5182c89 verified
preview code
|
Raw History Blame Contribute Delete
5.64 kB

DeepFense-CodecFake-W2V2-MLP β€” implementation notes

Plan: speech-spoof-bench/docs/plans/2026-11-07-eleven-model-batch-arena-submission.md (model #8).

Source

  • Weights: HF DeepFense/CodecFake_Wav2Vec2_MLP_NoAug_Seed42 @ c08bd2125c83e6c8fdc64566bfbfda18dff4dfb4, best_model.pth (3.79 GB, sha256 1d4bf44afd0f7e887266c8dc49875b0ddb60a07fb16279fa2d56598519ea3c1a = HF LFS oid)
    • config.yaml, in benchmarks/_ckpt_dl/DeepFense__CodecFake_Wav2Vec2_MLP_NoAug_Seed42/. The .pth is a training checkpoint: model_state (435 fp32 tensors, 317,522,306 params), optimizer_state (Adam), epoch=2, step=46298, best_metric=0.00306 (val EER fraction). No weight license on the Hub β†’ linked, not re-hosted.
  • Extraction: torch.load(weights_only=True) works (plain tensors/ints) β€” run once with python -I; model_state saved unchanged as model_state.safetensors (sha256 5c97bd4ae3a2075aa8df309bde8b7dd7f9ce540a50df969d4ff0ba7a74a560a5). The optimizer state is dropped.
  • Code: github.com/Yaselley/deepfense-framework @ 3f2fe1146c48ea4bacc480dd96d806b6c72c09cf (Apache-2.0), checked out to benchmarks/_ckpt_dl/Yaselley__deepfense-framework/ (not imported by the wrapper; only by source_parity.py).

Architecture (_net.py)

  • frontend.model: fairseq 0.12.2 Wav2Vec2Model built from the vendored xlsr2_300m_model_cfg.json (= state["cfg"]["model"] of the local W2V2-AASIST/xlsr2_300m.pt, merged onto Wav2Vec2Config defaults with merge_with_parent, as in W2V2-AASIST/_net.py). No .pt is loaded at runtime. Called as model(x, mask=False, features_only=True)["x"] (last layer, after the encoder's final layer_norm β€” layer_norm_first).
  • backend.projection_block = Linear(1024,128) β†’ LayerNorm(128) β†’ ReLU; mean over time (pool "mean").
  • losses.0.fc = Linear(128, 2).
  • Key mapping: the checkpoint was saved from real fairseq (it carries quantizer.*, project_q.*, final_proj.*), so all 435 tensors map 1:1 β†’ strict=True, no allow-list. Those 7 pre-training tensors are unused on the features_only path. (The repo's newer fairseq_local loader lacks those modules: strict=False reports exactly those 7 as unexpected, 0 missing.) params_millions 317.522306 counts them (as the checkpoint does).

Preprocessing / score (deepfense_codecfake_w2v2_mlp.py)

  • StandardDataset + transforms.load_audio / pad (test split: max_len 64000, pad_type repeat, random_pad false): soundfile float, mono mean, no normalisation anywhere (dataset, frontend: XLS-R normalize: false), first 64000 samples, shorter β†’ np.tile(x, ceil(64000/len))[:64000]. Empty clip β†’ zeros (not in source). No resampling. FP32, cuDNN TF32 off.
  • Score = logit1 βˆ’ logit0 (label_map bonafide=1), as specified in the plan; monotone in softmax p(bonafide), which is what the authors' simple.infer.py thresholds. Deviation note: the authors' test.py/trainer EER uses CrossEntropy.get_score = logits[:, 1] only (its docstring says "logit_bonafide βˆ’ logit_spoof" but the code returns the single logit). The two are not rank-equivalent; see the alt-score table below.
  • Source parity (source_parity.py, log logs_source_parity.txt): authors' ModularDetector (fairseq_local frontend from xlsr2_300m.pt, strict=False load of model_state) + their load_audio/pad vs this wrapper on 12 clips (6 J-SPAW_LA, 6 In-the-Wild): max |Ξ” logits| = 4.8e-6.

Batch sweep (sweep.py, 64000-sample input, TF32 off, 2026-11-07; logs_sweep_g{1,2}.txt)

card best notes
RTX 4070 Ti SUPER (idx 2) 123.9 utt/s @24 plateau 16–64
RTX 5060 Ti (idx 1) 48.5 utt/s @4 ~33 at bs β‰₯ 8 β†’ batch_size 4 on 5060

Sanity (PyTorch, RTX 5060 Ti idx 1)

  • Full J-SPAW_LA (results_pt/J-SPAW_LA, copied to results/J-SPAW_LA for submission): EER 0.625% (2397 trials, 0 skipped), ~46 utt/s end-to-end. Low but not inverted; Codecfake training data is codec re-synthesis, J-SPAW_LA spoofs are apparently easy for it.
  • Full ODSS (results_pt/ODSS): EER 13.17% (26,954 trials) vs self-reported 11.51% (+1.66 pt). The Arena ODSS build may differ from the authors' ODSS protocol (e.g. VCTK spoof-only subset, generator mix); not investigated further.
  • Alt score (alt_score_eer.py, logs_alt_score.txt): authors' test.py score logit1 vs ours logit1 βˆ’ logit0: J-SPAW_LA 0.625 / 0.625, ODSS 13.20 / 13.17 β†’ the score choice does not explain the ODSS gap and is immaterial.

TensorRT (trt_deepfense_codecfake_w2v2_mlp.py, sidecar trt_deepfense-codecfake-w2v2-mlp.json, TRT 10.16.1.11)

Adapted from Forensics-XLSR-Wild's script; _ExportNet returns (B,2), n_classes 2. FP16 fails the 0.9999 build-parity rule on both cards (Spearman 0.99899 on 4070, 0.99940 on 5060, 64 ITW clips) β†’ FP32 on both (FP16 engines deleted).

card precision best batch utt/s (sweep) build parity Spearman (64 ITW) J-SPAW EER J-SPAW Spearman vs PT / max Ξ”
RTX 4070 Ti SUPER (idx 2) FP32 32 182.4 0.99991 0.625 (=PT) 0.999980 / 0.0223
RTX 5060 Ti (idx 1) FP32 8 126.0 0.99995 0.625 (=PT) 0.999983 / 0.0317
Both pass ( Ξ”EER = 0, Spearman β‰₯ 0.999). Scripts: run_pt.sh, run_trt_export.sh, run_trt_jspaw.sh <gpu>, run_alt.sh.

Hosting

  • .gitignore excludes model_state.safetensors and *.onnx (unlicensed weights must not be re-hosted). meta.yaml checkpoint links the original HF best_model.pth @ c08bd21.
  • trained_on omitted (Codecfake is not an Arena dataset). Paper block = arXiv 2604.08450 (kheir2026deepfense).