schism-audio/htdemucs-coreml

HTDemucs — Core ML

Hybrid Transformer Demucs (4-stem: drums / bass / other / vocals; facebookresearch/demucs) for Core ML on Apple devices.

Converted from the same verified reference used by the schism-mlx MLX ports. Two variants per model:

File	Precision	Compute units	segment residual
HTDemucs_Core_fp16.mlpackage	FLOAT16	ALL (ANE)	−62.5 dB
HTDemucs_Core_fp32.mlpackage	FLOAT32	CPU+GPU	−123.8 dB

Verified on-device-equivalently via coremltools on an M5 Max, against the reference implementation on real audio. fp16 is ANE-eligible and recommended for iPhone / iPad; fp32 is the tight-parity fallback.

Download

.mlpackage bundles must be materialized as real files — the Core ML compiler rejects the symlinks that a default snapshot_download creates in the Hugging Face cache:

from huggingface_hub import snapshot_download
path = snapshot_download("schism-audio/htdemucs-coreml", local_dir="./htdemucs-coreml")

(or hf download schism-audio/htdemucs-coreml --local-dir ./htdemucs-coreml). Swift hosts downloading files directly are unaffected.

I/O contract

• input mag: (1, 4, 2048, 336) float32 — complex-as-channels STFT of one 7.8 s / 343980-sample segment ([c0_re, c0_im, c1_re, c1_im]), n_fft 4096, hop 1024, normalized 1/sqrt(n_fft), demucs framing (see test vectors note)
• input mix: (1, 2, 343980) float32 — the segment waveform
• output spec_out: (1, S, 4, 2048, 336) — CaC mask spectrogram → complex → iSTFT on host
• output time_out: (1, S, 2, 343980) — time-branch waveform; final stems = iSTFT(spec_out) + time_out
• full tracks: overlapping segments (25% overlap), triangular blending, centered padding pulled from surrounding audio

DSP frontend (host-side)

The Core ML graph contains the network only. The host implements the audio frontend and must match schism_mlx.audio numerically — test_vectors_*.npz in this repo holds deterministic input/output pairs (float32; match within ~1e-4 relative to be interchangeable with what these models were verified against). test_vectors_pipeline.npz additionally pins the full-track orchestration around the model call (chunk grid, blending, normalization) via a deterministic mock model — scaled-down sizes, same algorithm, generated by the shipped schism-mlx separate() / windowing code. test_vectors_integration.npz goes one step further: a full synthetic track plus the verified reference output, for end-to-end host validation (the schism-dsp integration tests run the published fp32 core through it). A validated Swift implementation (Accelerate; modules SchismDSP and SchismPipeline) is available at schism-audio/schism-dsp, tested against these exact vectors.

License

MIT, inherited from facebookresearch/demucs (Rouard, Massa, Défossez — Meta AI). Core ML conversion by schism-audio.