ash56/multimodal-asv-checkpoints

Multimodal ASV Checkpoints

This repository contains the released checkpoints for the GitHub project:

Ashigarg123/multimodal-speaker-verification

The code implements multi-utterance speaker verification models using acoustic, linguistic, and prosodic information, with experiments focused on speaker anonymization.

Installation

Clone and install the code repository:

git clone https://github.com/Ashigarg123/multimodal-speaker-verification.git
cd multimodal-speaker-verification
uv venv --python 3.10
source .venv/bin/activate
uv pip install -e .[dev,whisper]

Download Checkpoints

Use huggingface_hub to download a checkpoint locally:

from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="ash56/multimodal-asv-checkpoints",
    filename="audio_only_query_attention/epoch_5.pt",
)
print(checkpoint_path)

You can also download the full repository snapshot:

from huggingface_hub import snapshot_download

repo_dir = snapshot_download("ash56/multimodal-asv-checkpoints")
print(repo_dir)

The WavLM-Large frontend checkpoint is not stored here. Follow the setup instructions in the GitHub repository README and set model.audio_encoder.wavlm_checkpoint in the selected config to your local WavLM checkpoint path.

Citation

If you use these checkpoints or the associated code, please cite:

@article{garg2026multimodal,
  title={Multimodal Speaker Verification as a Threat to Speaker Anonymization},
  author={Garg, Ashi and Aggazzotti, Cristina and Garc{\'\i}a-Perera, Leibny Paola and Andrews, Nicholas},
  journal={arXiv preprint arXiv:2607.19636},
  year={2026}
}