File size: 5,238 Bytes
6fd352c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | ---
language:
- en
pipeline_tag: audio-classification
datasets:
- CodecFake/CodecFake_Plus_Dataset
tags:
- audio
- speech
- deepfake-detection
- audio-deepfake-detection
- anti-spoofing
- source-tracing
- neural-audio-codec
- codecfake
---
# CodecFake Source Tracing Checkpoints
[](https://arxiv.org/abs/2505.12994)
[](https://arxiv.org/abs/2506.07294)
[](https://github.com/ResponsibleGenAI/CodecFake-Source-Tracing)
[](https://huggingface.co/datasets/CodecFake/CodecFake_Plus_Dataset)
[](https://responsiblegenai.github.io/CodecFake-Plus-Dataset/)
Pretrained model checkpoints for our source tracing work on codec-based deepfake speech. Training and inference code is on [GitHub](https://github.com/ResponsibleGenAI/CodecFake-Source-Tracing).
🏆 [Towards Generalized Source Tracing for Codec-Based Deepfake Speech](https://arxiv.org/abs/2506.07294) received a **Best Student Paper Nomination (Top 2.5%)** at IEEE ASRU 2025.
## Tasks and Training Subsets
| Task | Meaning |
|---|---|
| `BIN` | Binary spoof detection |
| `VQ` | Vector quantization source tracing |
| `AUX` | Auxiliary objective source tracing |
| `DEC` | Decoder type source tracing |
| Subset | Sampling in the training data |
|---|---|
| `vq` | MVQ : SVQ : SQ = 1 : 1 : 1 |
| `aux` | None : Semantic Distillation : Disentanglement = 1 : 1 : 1 |
| `dec` | Time : Frequency = 1 : 1 |
## Checkpoints
### Wav2Vec2-AASIST
Files are in `model_checkpoints/Wav2Vec2_baseline/`.
| Model | Learning | Task | Trained on | Files |
|---|---|---|---|---|
| `S_BIN` | Single-task | BIN | vq / aux / dec | `S_BIN_VQ_bal.pth`, `S_BIN_AUX_bal.pth`, `S_BIN_DEC_bal.pth` |
| `S_VQ` | Single-task | VQ | vq | `S_VQ.pth` |
| `S_AUX` | Single-task | AUX | aux | `S_AUX.pth` |
| `S_DEC` | Single-task | DEC | dec | `S_DEC.pth` |
| `D_VQ` | Dual-task | BIN / VQ | vq | `D_VQ.pth` |
| `D_AUX` | Dual-task | BIN / AUX | aux | `D_AUX.pth` |
| `D_DEC` | Dual-task | BIN / DEC | dec | `D_DEC.pth` |
| `M1` | Multi-task | BIN / VQ / AUX / DEC | vq / aux / dec | `M1_VQ_bal.pth`, `M1_AUX_bal.pth`, `M1_DEC_bal.pth` |
| `M2` | Multi-task | VQ / AUX / DEC | vq / aux / dec | `M2_VQ_bal.pth`, `M2_AUX_bal.pth`, `M2_DEC_bal.pth` |
### SAST Net
Files are in `model_checkpoints/SAST_Net/`.
| Task | Trained on | Files |
|---|---|---|
| BIN | vq / aux / dec | `SAST_Net_BIN_VQ_bal.pth`, `SAST_Net_BIN_AUX_bal.pth`, `SAST_Net_BIN_DEC_bal.pth` |
| VQ | vq | `SAST_Net_VQ.pth` |
| AUX | aux | `SAST_Net_AUX.pth` |
| DEC | dec | `SAST_Net_DEC.pth` |
### Pretrained weights
Files are in `model_checkpoints/Pretrain_weight/`.
| File | Description | Used by |
|---|---|---|
| `xlsr2_300m.pt` | Wav2Vec2 pretrained weight | Wav2Vec2-AASIST, SAST Net |
| `mae_pretrained_base.pth` | AudioMAE pretrained on AudioSet | SAST Net |
| `tuned_weight.pth` | Wav2Vec2-AASIST on CodecFake+ | SAST Net |
## Usage
Download a checkpoint:
```python
from huggingface_hub import hf_hub_download
checkpoint = hf_hub_download(
repo_id="CodecFake/CodecFake_Source_Tracing",
filename="model_checkpoints/SAST_Net/SAST_Net_VQ.pth",
)
```
Setup, inference, and training commands are in the [GitHub README](https://github.com/ResponsibleGenAI/CodecFake-Source-Tracing).
> **Note:** The `metadata/` files in the GitHub repository use the file names from our internal copy of the dataset. The public [CodecFake+ release](https://huggingface.co/datasets/CodecFake/CodecFake_Plus_Dataset) uses a different layout and different CoSG file names, so the paths need to be adapted before running the scripts.
## Citation
```bibtex
@inproceedings{chen2025towards,
title = {Towards Generalized Source Tracing for Codec-Based Deepfake Speech},
author = {Chen, Xuanjun and Lin, I-Ming and Zhang, Lin and Wu, Haibin and Lee, Hung-yi and Jang, Jyh-Shing Roger},
booktitle = {2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)},
pages = {1--8},
year = {2025},
doi = {10.1109/ASRU65441.2025.11434771}
}
@inproceedings{chen2025codec,
title = {Codec-Based Deepfake Source Tracing via Neural Audio Codec Taxonomy},
author = {Chen, Xuanjun and Lin, I-Ming and Zhang, Lin and Du, Jiawei and Wu, Haibin and Lee, Hung-yi and Jang, Jyh-Shing Roger},
booktitle = {{INTERSPEECH 2025}},
pages = {1538--1542},
year = {2025},
doi = {10.21437/Interspeech.2025-1297}
}
@article{chen2026codecfake+,
title = {{CodecFake+}: Codec-Based Resynthesized Data as a Proxy for Detecting {CodecFake} Speech},
author = {Chen, Xuanjun and Du, Jiawei and Wu, Haibin and Zhang, Lin and Lin, I-Ming and Chiu, I-Hsiang and Ren, Wenze and Tseng, Yuan and Tsao, Yu and Jang, Jyh-Shing Roger and Lee, Hung-yi},
journal = {IEEE Transactions on Audio, Speech and Language Processing},
volume = {34},
pages = {2929--2944},
year = {2026},
doi = {10.1109/TASLPRO.2026.3692291}
}
```
|