| --- |
| license: apache-2.0 |
| tags: |
| - audio |
| - deepfake-detection |
| - anti-spoofing |
| - lora |
| - qwen |
| - whisper |
| - wav2vec |
| - opensmile |
| language: |
| - en |
| --- |
| |
| # AudioLLM-Deepfake-Detection — Results Hub |
|
|
| Best-run checkpoints, evaluation CSVs/JSONs, and aggregated metrics for the **DeepFense AudioLLM Deepfake Detection** project. |
|
|
| **Hub repo:** [01Yassine/AudioLLM-Deepfake-Detection](https://huggingface.co/models/01Yassine/AudioLLM-Deepfake-Detection) |
|
|
| ## Contents (~283 GB) |
|
|
| | Folder | Description | Size | |
| |--------|-------------|------| |
| | `Suite/` | Baseline Whisper/Wav2Vec + Qwen (LoRA α=16/64/128/256, NoLoRA) | ~82 GB | |
| | `OpenSmile/` | OpenSmile **before** stage | ~54 GB | |
| | `OpenSmile-After/` | OpenSmile **after** stage (best overall) | ~62 GB | |
| | `EAT/` | EAT encoder experiments | ~27 GB | |
| | `Weighted/` | Layer-weighted fusion | ~59 GB | |
| | `DAC-6-Lora-Qwen0.5B/` | DAC tokenizer baseline | ~8 MB | |
| | `SpeechTokenizer-4-Lora-Qwen0.5B/` | SpeechTokenizer baseline | ~18 MB | |
| | `Qwen-Omni-3B-lora-full/` | Qwen2.5-Omni end-to-end | ~422 MB | |
| | `Qwen-Omni-3B-lora-opensmile/` | Qwen2.5-Omni + OpenSmile | ~462 MB | |
|
|
| Each run folder includes: `best_run_meta.json`, per-dataset eval CSVs, metrics JSON (with EER), and checkpoints (`lora_best/`, `checkpoint_best.pt`, etc.). |
|
|
| ## Aggregated metrics (machine-readable) |
|
|
| | File | Description | |
| |------|-------------| |
| | `all_results_table.json` | Full nested table (89 runs): Macro F1, Accuracy, EER | |
| | `all_results_table.csv` | Flat CSV for spreadsheets / LaTeX | |
| | `all_results_table.md` | Markdown tables by experiment family | |
|
|
| ## Best overall model |
|
|
| **OpenSmile-After / Lora-256 / unfrozen / Whisper / Qwen-0.5B / α=256** |
|
|
| - Avg Macro F1: **94.42%** |
| - Avg Accuracy: **95.11%** |
| - Avg EER: **5.36%** |
| - Path: `OpenSmile-After/Lora-256/unfrozen/whisper/Qwen-0.5B/` |
|
|
| --- |
|
|
| # Results Summary (local documentation) |
|
|
| Unified table of **best runs** across all experiment families. |
|
|
| ## Metrics |
|
|
| | Metric | Description | |
| |--------|-------------| |
| | **Macro F1** | Unweighted average of Fake-class and Real-class F1 (equal weight per class) | |
| | **Accuracy** | Fraction of correct predictions | |
| | **EER** | Equal Error Rate from `score_fake` vs ground truth (lower is better); from metrics JSON when available | |
|
|
| ## Datasets |
|
|
| - **asv19_test** — ASVspoof 2019 LA eval |
| - **itw** — In-The-Wild |
| - **la21** — ASVspoof 2021 LA eval |
| - **mlaad_en** — MLAAD English |
|
|
| Averages (`avg_*`) are computed over evaluated datasets for each run (typically 4/4). |
|
|
| ## Experiment Families |
|
|
| | Family | Description | |
| |--------|-------------| |
| | **Suite** | Baseline Whisper/Wav2Vec + Qwen LLM (LoRA α=16/64/128/256, frozen/unfrozen, NoLoRA) | |
| | **OpenSmile** | OpenSmile features injected **before** LLM (stage 1) | |
| | **OpenSmile-After** | OpenSmile features injected **after** audio encoder (stage 2) | |
| | **EAT** | EAT audio encoder + Qwen | |
| | **Weighted** | Layer-weighted fusion of Whisper/Wav2Vec representations | |
| | **DAC-6-Lora-Qwen0.5B** | DAC tokenizer (6 codebooks) + Qwen-0.5B LoRA | |
| | **SpeechTokenizer-4-Lora-Qwen0.5B** | SpeechTokenizer + Qwen-0.5B LoRA | |
| | **Qwen-Omni-3B-lora-full** | Qwen2.5-Omni-3B end-to-end LoRA | |
| | **Qwen-Omni-3B-lora-opensmile** | Qwen2.5-Omni-3B LoRA + OpenSmile | |
|
|
| ## Best Overall Models |
|
|
| ### Highest average Macro F1 |
| - **OpenSmile-After / Lora-256 / unfrozen / whisper / Qwen-0.5B / α=256** — **94.42%** macro F1, 95.11% accuracy, 5.36% EER |
| - Path: `results/OpenSmile-After/Lora-256/unfrozen/whisper/Qwen-0.5B` |
|
|
| ### Lowest average EER |
| - **OpenSmile-After / Lora-256 / unfrozen / whisper / Qwen-0.5B / α=256** — **5.36%** EER, 94.42% macro F1, 95.11% accuracy |
| - Path: `results/OpenSmile-After/Lora-256/unfrozen/whisper/Qwen-0.5B` |
|
|
| ## Files |
|
|
| | File | Format | Use | |
| |------|--------|-----| |
| | `all_results_table.json` | Nested JSON | Machine-readable; full per-dataset breakdown | |
| | `all_results_table.csv` | Flat CSV | Spreadsheet / LaTeX table generation | |
| | `all_results_table.md` | Markdown tables | Human-readable, grouped by family | |
|
|
| ## Notes |
|
|
| - Some runs borrow missing eval splits (documented in JSON `notes` / `borrowed_or_approximate`). |
| - OpenSmile **before** stage: LoRA α=16 and Lora-128 only for Qwen-0.5B; NoLoRA all sizes. |
| - OpenSmile **after** stage: full LoRA α sweep (0.5B) + α=16 for 3B/7B. |
| - EER requires score columns in eval CSV; if metrics JSON missing, EER computed from CSV. |
|
|
