davidbraun/fm-synapse-dx7-gnn

FM-SynAPSE: DX7-GNN

Joint embeddings of DX7 audio and DX7 presets, so an audio query retrieves presets by cosine similarity. Both modalities live in one space, trained with the SLAP objective (BYOL-style, non-contrastive, no negative samples).

The parameter encoder is a DX7-GNN: a patch is a 6-operator graph, and messages pass along the algorithm's modulation and feedback edges, with each operator's state gated by its output level. Because the message-passing weights are shared across all nodes and layers, the encoder handles routing topologies it never saw during training — which is what the held-out variant below measures.

This repository holds two trained models, one per data split. Each is a complete SLAP model: the PANNs audio encoder, the DX7-GNN parameter encoder, and both arms' projector and predictor heads.

Subfolder	Split	Audio→Preset R@1	R@10	MRR	Modality gap
held-out	16 odd algorithms train / 8 even test	52.2%	88.5%	0.652	0.0446
80-10-10	random 80/10/10 over all 32 algorithms	86.1%	99.5%	0.917	0.0293

Galleries differ: 4,096 presets for held-out, 3,144 for 80-10-10. Each query's audio render has exactly one correct preset in its gallery.

Usage

pip install "synapse[hf] @ git+https://github.com/DBraun/SynAPSE"

from audiotree import AudioTree
from jax import numpy as jnp
from synapse import SynAPSE

model = SynAPSE.from_pretrained("davidbraun/fm-synapse-dx7-gnn", subfolder="held-out")

B = 1
batch = AudioTree(
    jnp.zeros((B, 1, 176400)),                    # ~4 s at 44.1 kHz
    sample_rate=44100,
    extras={
        "params": jnp.zeros((B, 145)),            # dexed Preset.to_array()
        "algorithm": jnp.zeros((B,), jnp.int32),  # 0..31
    },
)

out = model(batch)
out.audio_output.latent            # [B, 768] audio encoder output (y_A)
out.audio_output.projection        # [B, 384] audio projection (z_A)
out.audio_output.prediction        # [B, 384] audio prediction (q_A)
out.parameter_output.latent        # [B, 512] DX7-GNN output (y_P)
out.parameter_output.prediction    # [B, 384] preset prediction (q_P)

Retrieval scores are cosine(q_A, q_P), following SLAP. Cosine similarity on the projections works too, but the paper reports the prediction-space distances.

Training

• Data: DX7AllTheWeb, deduplicated by exact parameter values and filtered to 31,443 presets whose 2-second renders exceed −40 LUFS.
• Audio: rendered on the fly with dexed-py — 4 s (3 s note + 1 s release), MIDI C4, velocity 85, 44.1 kHz downsampled to 22.05 kHz.
• Audio encoder: AFx-Rep's CNN14 variation of PANNs, trained from scratch (82.8M parameters), 768-dim output.
• Parameter encoder: DX7-GNN, 9 message-passing layers, hidden dim 384, 3.38M parameters.
• Augmentation: operator swapping (20% per non-silent operator) and parameter noise (1% continuous, 2% categorical).
• Optimization: AdamW, effective batch size 256, LR 2e-4 constant with 1000-step warmup, gradient clipping 3.0, on four NVIDIA L40 GPUs.

Ablations in the paper show operator swapping is the decisive augmentation (removing it drops R@1 from 52.2% to 13.5%) and that the specific routing matters (a fully connected graph drops R@1 to 30.9%).

Limitations

• Trained exclusively on DX7 FM synthesis at a single pitch and velocity (C4, 85). Retrieval quality at other pitches, velocities, or on non-DX7 audio is unmeasured.
• DX7 symmetries mean distinct presets can render identical audio, so some retrieval "errors" are perceptually correct.
• Retrieval quality has not been assessed in a user study.
• The held-out model is the interesting one for topology generalization; the 80-10-10 model saw every algorithm during training and should not be read as evidence about unseen topologies.

Citation

@inproceedings{braun2026fmsynapse,
  title     = {{FM} Synthesizer Audio-Parameter Shared Embeddings},
  author    = {Braun, David and Finkelstein, Adam},
  booktitle = {Proc. of the 29th Int. Conference on Digital Audio Effects (DAFx26)},
  address   = {Cambridge, MA, USA},
  year      = {2026},
  month     = {September},
}