| --- |
| language: |
| - en |
| pipeline_tag: audio-classification |
| datasets: |
| - CodecFake/CodecFake_Plus_Dataset |
| tags: |
| - audio |
| - speech |
| - deepfake-detection |
| - audio-deepfake-detection |
| - anti-spoofing |
| - source-tracing |
| - neural-audio-codec |
| - codecfake |
| --- |
| |
| # CodecFake Source Tracing Checkpoints |
|
|
| [](https://arxiv.org/abs/2505.12994) |
| [](https://arxiv.org/abs/2506.07294) |
| [](https://github.com/ResponsibleGenAI/CodecFake-Source-Tracing) |
| [](https://huggingface.co/datasets/CodecFake/CodecFake_Plus_Dataset) |
| [](https://responsiblegenai.github.io/CodecFake-Plus-Dataset/) |
|
|
| Pretrained model checkpoints for our source tracing work on codec-based deepfake speech. Training and inference code is on [GitHub](https://github.com/ResponsibleGenAI/CodecFake-Source-Tracing). |
|
|
| 🏆 [Towards Generalized Source Tracing for Codec-Based Deepfake Speech](https://arxiv.org/abs/2506.07294) received a **Best Student Paper Nomination (Top 2.5%)** at IEEE ASRU 2025. |
|
|
| ## Tasks and Training Subsets |
|
|
| | Task | Meaning | |
| |---|---| |
| | `BIN` | Binary spoof detection | |
| | `VQ` | Vector quantization source tracing | |
| | `AUX` | Auxiliary objective source tracing | |
| | `DEC` | Decoder type source tracing | |
|
|
| | Subset | Sampling in the training data | |
| |---|---| |
| | `vq` | MVQ : SVQ : SQ = 1 : 1 : 1 | |
| | `aux` | None : Semantic Distillation : Disentanglement = 1 : 1 : 1 | |
| | `dec` | Time : Frequency = 1 : 1 | |
|
|
| ## Checkpoints |
|
|
| ### Wav2Vec2-AASIST |
|
|
| Files are in `model_checkpoints/Wav2Vec2_baseline/`. |
|
|
| | Model | Learning | Task | Trained on | Files | |
| |---|---|---|---|---| |
| | `S_BIN` | Single-task | BIN | vq / aux / dec | `S_BIN_VQ_bal.pth`, `S_BIN_AUX_bal.pth`, `S_BIN_DEC_bal.pth` | |
| | `S_VQ` | Single-task | VQ | vq | `S_VQ.pth` | |
| | `S_AUX` | Single-task | AUX | aux | `S_AUX.pth` | |
| | `S_DEC` | Single-task | DEC | dec | `S_DEC.pth` | |
| | `D_VQ` | Dual-task | BIN / VQ | vq | `D_VQ.pth` | |
| | `D_AUX` | Dual-task | BIN / AUX | aux | `D_AUX.pth` | |
| | `D_DEC` | Dual-task | BIN / DEC | dec | `D_DEC.pth` | |
| | `M1` | Multi-task | BIN / VQ / AUX / DEC | vq / aux / dec | `M1_VQ_bal.pth`, `M1_AUX_bal.pth`, `M1_DEC_bal.pth` | |
| | `M2` | Multi-task | VQ / AUX / DEC | vq / aux / dec | `M2_VQ_bal.pth`, `M2_AUX_bal.pth`, `M2_DEC_bal.pth` | |
|
|
| ### SAST Net |
|
|
| Files are in `model_checkpoints/SAST_Net/`. |
|
|
| | Task | Trained on | Files | |
| |---|---|---| |
| | BIN | vq / aux / dec | `SAST_Net_BIN_VQ_bal.pth`, `SAST_Net_BIN_AUX_bal.pth`, `SAST_Net_BIN_DEC_bal.pth` | |
| | VQ | vq | `SAST_Net_VQ.pth` | |
| | AUX | aux | `SAST_Net_AUX.pth` | |
| | DEC | dec | `SAST_Net_DEC.pth` | |
|
|
| ### Pretrained weights |
|
|
| Files are in `model_checkpoints/Pretrain_weight/`. |
|
|
| | File | Description | Used by | |
| |---|---|---| |
| | `xlsr2_300m.pt` | Wav2Vec2 pretrained weight | Wav2Vec2-AASIST, SAST Net | |
| | `mae_pretrained_base.pth` | AudioMAE pretrained on AudioSet | SAST Net | |
| | `tuned_weight.pth` | Wav2Vec2-AASIST on CodecFake+ | SAST Net | |
|
|
| ## Usage |
|
|
| Download a checkpoint: |
|
|
| ```python |
| from huggingface_hub import hf_hub_download |
| |
| checkpoint = hf_hub_download( |
| repo_id="CodecFake/CodecFake_Source_Tracing", |
| filename="model_checkpoints/SAST_Net/SAST_Net_VQ.pth", |
| ) |
| ``` |
|
|
| Setup, inference, and training commands are in the [GitHub README](https://github.com/ResponsibleGenAI/CodecFake-Source-Tracing). |
|
|
| > **Note:** The `metadata/` files in the GitHub repository use the file names from our internal copy of the dataset. The public [CodecFake+ release](https://huggingface.co/datasets/CodecFake/CodecFake_Plus_Dataset) uses a different layout and different CoSG file names, so the paths need to be adapted before running the scripts. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{chen2025towards, |
| title = {Towards Generalized Source Tracing for Codec-Based Deepfake Speech}, |
| author = {Chen, Xuanjun and Lin, I-Ming and Zhang, Lin and Wu, Haibin and Lee, Hung-yi and Jang, Jyh-Shing Roger}, |
| booktitle = {2025 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU)}, |
| pages = {1--8}, |
| year = {2025}, |
| doi = {10.1109/ASRU65441.2025.11434771} |
| } |
| |
| @inproceedings{chen2025codec, |
| title = {Codec-Based Deepfake Source Tracing via Neural Audio Codec Taxonomy}, |
| author = {Chen, Xuanjun and Lin, I-Ming and Zhang, Lin and Du, Jiawei and Wu, Haibin and Lee, Hung-yi and Jang, Jyh-Shing Roger}, |
| booktitle = {{INTERSPEECH 2025}}, |
| pages = {1538--1542}, |
| year = {2025}, |
| doi = {10.21437/Interspeech.2025-1297} |
| } |
| |
| @article{chen2026codecfake+, |
| title = {{CodecFake+}: Codec-Based Resynthesized Data as a Proxy for Detecting {CodecFake} Speech}, |
| author = {Chen, Xuanjun and Du, Jiawei and Wu, Haibin and Zhang, Lin and Lin, I-Ming and Chiu, I-Hsiang and Ren, Wenze and Tseng, Yuan and Tsao, Yu and Jang, Jyh-Shing Roger and Lee, Hung-yi}, |
| journal = {IEEE Transactions on Audio, Speech and Language Processing}, |
| volume = {34}, |
| pages = {2929--2944}, |
| year = {2026}, |
| doi = {10.1109/TASLPRO.2026.3692291} |
| } |
| ``` |
|
|
