sensein/driftse-distilhubert-three-layers

DriftSE — DistilHuBERT, three layers, with-z (mirror)

Mirror of the released checkpoint for Speech Enhancement Based on Drifting Models (Xu, Caviedes-Nozal, Kleijn, Yan & Olsson, Interspeech 2026, oral), arXiv:2604.24199.

Held so that senselab's speech_enhancement DriftSE backend can pin a revision and a content hash. Nothing here is original work of the Senseable Intelligence Group.

Provenance

• Upstream code: https://github.com/LiangXu123/DriftSE, pinned at 695a64db187500fa0d7bae23912680bd5d4df613.
• Weights: the authors' own Google Drive release linked from the upstream README (logs/distillhubert_three_layers_with_z/last.ckpt).
• config.json is upstream's config/with_z/v2_drift2_distillhubert_three_layers.json at the same commit.

Licence status — unresolved

The upstream repository has no LICENSE file and no licence statement, so the default is all rights reserved. It is also built on SGMSE+ (MIT) without carrying that statement forward. A request for an explicit licence has been opened upstream; this mirror stays private until it is answered. Do not make it public without that answer.

Contents

File	What it is
last.ckpt	logs/distillhubert_three_layers_with_z/last.ckpt. Pairs with config.json. This is the variant senselab's backend pins.
config.json	Upstream config/with_z/v2_drift2_distillhubert_three_layers.json.
last_pesq_sisdr_ccmse.ckpt	logs/distillhubert_three_layers_pesq_sisdr_ccmse_with_z/last.ckpt — the DriftSE† row, jointly trained with auxiliary PESQ and SI-SDR losses, and the best published numbers on both benchmark tables (VB-DMD PESQ 3.45 / SI-SDR 20.60; DNS-2020 WV-MOS 2.65 / SCOREQ 2.97). Pairs with config_pesq_sisdr_ccmse.json.
config_pesq_sisdr_ccmse.json	Upstream config/with_z/v2_drift2_distillhubert_three_layers_pesq_sisdr_ccmse.json.

Both checkpoints were pulled from the same Google Drive logs/ archive, which holds seven variants in total; these two are the ones worth keeping. They are mirrored verbatim, so each still contains its optimizer and scheduler state — that is most of the ~1.1 GB. Stripping it would shrink the download but would make these derived artifacts rather than a faithful mirror.

Notes for users

Inference is a single network evaluation (1 NFE). The SSL encoders named in the paper (HuBERT / WavLM / DistilHuBERT) are the training signal for the drifting field and are not required at inference, so the upstream latent_ckpt/ archive is not mirrored here.

model vs ema. Each checkpoint contains both a model and an ema state dict. Upstream's enhancement.py loads checkpoint["model"], so that is what senselab loads and what reproduces the published numbers — even though EMA weights usually score better, and even though the sibling unasdiff codebase does the opposite and loads its ema copy. If you switch to ema, you are no longer running what the paper reports; measure before believing it.

Citation

@inproceedings{xu2026driftse,
  author    = {Liang Xu and Diego Caviedes-Nozal and W. Bastiaan Kleijn and Longfei Felix Yan and Rasmus Kongsgaard Olsson},
  title     = {Speech Enhancement Based on Drifting Models},
  booktitle = {Proc. Interspeech 2026},
  year      = {2026}
}