github-issues/T-005.md

T-005: nih-plug VST3/CLAP Scaffold — Audio Passthrough, Shared Core Crate, Bundler

Type: Task
Phase: 0 — Foundation
Autonomy: agent:human-led (A3) — Agent implements the full scaffold. Human must confirm the three identity constants marked [HUMAN DECISION] before the PR is merged. Everything else is fully specified.
Stack: stack:rust
Version: v0.1
Iteration: iter-1
Effort: M (1–2 days)

---

⚠️ Agent Scope: Create the plugin/ crate with nih-plug, implement stereo audio passthrough, set up the xtask bundler, and verify the plugin loads in Reaper. Do not add any model inference, IPC calls, or generation logic — those arrive in T-032 (Perf RNN VST), T-065, T-089 (Magenta RT VST). Parameters beyond a single placeholder gain are also out of scope.

---

Context

Synesthesia ships as both a standalone Tauri desktop app and a VST3/CLAP plugin. Both compile from the same synesthesia-core library crate. The plugin uses nih-plug — MIT-licensed, Rust-native, produces VST3 + CLAP from the same source, no JUCE commercial license required.

Why not JUCE: JUCE's GPL licence requires open-sourcing your plugin or purchasing a commercial licence. nih-plug is MIT, ships as a single Rust crate, and the community has validated it against Reaper, Ableton, Bitwig, and FL Studio.

Dual-target build model:

synesthesia-core   (library crate)
      ├── kansas/           ← Tauri app depends on this
      └── plugin/           ← nih-plug plugin depends on this

synesthesia-core is the only place shared business logic lives. Neither kansas/ nor plugin/ duplicates code. T-006 scaffolds synesthesia-core's content; T-005 just adds plugin/ as a consumer of it.

---

Human Decisions Required Before Merge

The following three constants in plugin/src/lib.rs must be reviewed and confirmed by the human before the PR is approved. The values below are agent defaults — the human changes them if desired:

// [HUMAN DECISION 1] — VST3 Class ID
// Must be exactly 16 bytes and globally unique per plugin.
// This ID is baked into every DAW project that uses the plugin.
// Changing it after release breaks saved sessions.
// Default (agent sets this): ASCII-padded plugin name
const VST3_CLASS_ID: [u8; 16] = *b"Synesthesia_Plug";

// [HUMAN DECISION 2] — Plugin metadata shown in DAW
const VENDOR: &str = "Ash";
const URL: &str = "https://github.com/ashiedu/synesthesia";

// [HUMAN DECISION 3] — CLAP feature tags (affects DAW browser categorization)
const CLAP_FEATURES: &[ClapFeature] = &[
    ClapFeature::Instrument,
    ClapFeature::Synthesizer,
    ClapFeature::Stereo,
];

Open a PR comment with [HUMAN DECISION] in the title containing the proposed values. Do not merge until the human replies with explicit confirmation or updated values.

---

Prerequisites

• T-001 merged — workspace Cargo.toml with edition = "2024", crates/synesthesia-core/ exists as empty lib
• cargo check --workspace passes before starting

Note: T-004 (CPAL audio) and T-006 (synesthesia-core scaffold) do not need to be merged first. T-005 only requires the workspace to exist.

---

New Dependencies

nih-plug is not on crates.io — it is a git dependency. The wildcard version policy does not apply to git deps; use the git URL directly.

Add to root Cargo.toml [workspace.dependencies]:

# nih-plug — git only, not on crates.io
nih_plug      = { git = "https://github.com/robbert-vdh/nih-plug.git", features = ["vst3", "clap"] }
nih_plug_xtask = { git = "https://github.com/robbert-vdh/nih-plug.git" }

Cargo.lock will pin these to the HEAD commit at first cargo build. Commit Cargo.lock so the pinned revision is reproducible.

---

Workspace Changes (Cargo.toml)

[workspace]
members = [
    "kansas",
    "crates/*",
    "plugin",
    "xtask",        # cargo xtask bundler
]
resolver = "2"

---

File Structure After This Task

synesthesia/
├── Cargo.toml               ← updated: adds plugin, xtask members
├── plugin/
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs           ← Plugin impl (passthrough + params stub)
│       └── params.rs        ← SynesthesiaParams (single gain knob)
└── xtask/
    ├── Cargo.toml
    └── src/
        └── main.rs          ← delegates to nih_plug_xtask::main()

---

Implementation

plugin/Cargo.toml

[package]
name    = "synesthesia-plugin"
version = "0.1.0"
edition.workspace = true

# REQUIRED: cdylib produces the .dll/.so/.dylib the DAW loads
[lib]
name      = "synesthesia_plugin"
crate-type = ["cdylib"]

[dependencies]
nih_plug         = { workspace = true }
synesthesia-core = { path = "../crates/synesthesia-core" }

plugin/src/params.rs

use nih_plug::prelude::*;

/// Plugin parameters.
/// T-005: single placeholder gain knob — proves the parameter system compiles.
/// T-032 (Perf RNN) adds: Temperature, Primer, BPM Sync
/// T-065 (Perf RNN standalone) adds: channel routing
/// T-089 (Magenta RT) adds: Style Blend 1–6
#[derive(Params)]
pub struct SynesthesiaParams {
    /// Output gain in dB. Range –96 to +6. Default 0 dB.
    /// This is a smoke-test parameter; it is not used in T-005's passthrough.
    #[id = "gain"]
    pub gain: FloatParam,
}

impl Default for SynesthesiaParams {
    fn default() -> Self {
        Self {
            gain: FloatParam::new(
                "Gain",
                util::db_to_gain(0.0),
                FloatRange::Skewed {
                    min: util::db_to_gain(-96.0),
                    max: util::db_to_gain(6.0),
                    factor: FloatRange::gain_skew_factor(-96.0, 6.0),
                },
            )
            .with_smoother(SmoothingStyle::Logarithmic(50.0))
            .with_unit(" dB")
            .with_value_to_string(formatters::v2s_f32_gain_to_db(2))
            .with_string_to_value(formatters::s2v_f32_gain_to_db()),
        }
    }
}

plugin/src/lib.rs

use std::sync::Arc;
use nih_plug::prelude::*;

mod params;
use params::SynesthesiaParams;

// Unused in T-005; imported to verify synesthesia-core links correctly.
// T-006 populates this crate; for now it's just a link-time check.
use synesthesia_core as _core;

// ─── Plugin struct ────────────────────────────────────────────────────────────

pub struct SynesthesiaPlugin {
    params: Arc<SynesthesiaParams>,
    sample_rate: f32,
}

impl Default for SynesthesiaPlugin {
    fn default() -> Self {
        Self {
            params: Arc::new(SynesthesiaParams::default()),
            sample_rate: 44_100.0,
        }
    }
}

// ─── nih-plug Plugin trait ────────────────────────────────────────────────────

impl Plugin for SynesthesiaPlugin {
    // ── Identity ──────────────────────────────────────────────────────────────

    const NAME:    &'static str = "Synesthesia";
    const VENDOR:  &'static str = "Ash";            // [HUMAN DECISION 2]
    const URL:     &'static str = "https://github.com/ashiedu/synesthesia"; // [HUMAN DECISION 2]
    const EMAIL:   &'static str = "";
    const VERSION: &'static str = env!("CARGO_PKG_VERSION");

    // ── Audio I/O ─────────────────────────────────────────────────────────────

    /// Stereo in, stereo out. The DAW may also call us with no inputs (instrument mode).
    const AUDIO_IO_LAYOUTS: &'static [AudioIOLayout] = &[
        AudioIOLayout {
            main_input_channels:  NonZeroU32::new(2),
            main_output_channels: NonZeroU32::new(2),
            aux_input_ports:  &[],
            aux_output_ports: &[],
            names: PortNames::const_default(),
        },
        // Instrument mode — no audio input, stereo output
        AudioIOLayout {
            main_input_channels:  None,
            main_output_channels: NonZeroU32::new(2),
            aux_input_ports:  &[],
            aux_output_ports: &[],
            names: PortNames::const_default(),
        },
    ];

    // ── MIDI ──────────────────────────────────────────────────────────────────
    // T-068 (MIDI output) activates BasicMidi output.
    // T-005: accept MIDI so DAWs route notes to us; output is off until T-068.
    const MIDI_INPUT:  MidiConfig = MidiConfig::Basic;
    const MIDI_OUTPUT: MidiConfig = MidiConfig::None;

    type SysExMessage  = ();
    type BackgroundTask = ();

    // ── Parameters ────────────────────────────────────────────────────────────

    fn params(&self) -> Arc<dyn Params> {
        self.params.clone()
    }

    // ── Lifecycle ─────────────────────────────────────────────────────────────

    fn initialize(
        &mut self,
        _audio_io_layout: &AudioIOLayout,
        buffer_config: &BufferConfig,
        _context: &mut impl InitContext<Self>,
    ) -> bool {
        self.sample_rate = buffer_config.sample_rate;
        // T-046+: initialize ONNX runtime here when model inference is added
        true
    }

    fn reset(&mut self) {
        // T-062+: reset generator state (e.g., Perf RNN LSTM hidden state) here
    }

    // ── Audio processing ──────────────────────────────────────────────────────

    fn process(
        &mut self,
        buffer: &mut Buffer,
        _aux: &mut AuxiliaryBuffers,
        _context: &mut impl ProcessContext<Self>,
    ) -> ProcessStatus {
        // T-005: passthrough with gain applied.
        // T-032 replaces this with Perf RNN MIDI generation.
        // T-065 adds Magenta RT audio generation.
        // T-095 adds DDSP timbre transfer.

        let gain = self.params.gain.smoothed.next();

        for channel_samples in buffer.iter_samples() {
            for sample in channel_samples {
                *sample *= gain;
            }
        }

        ProcessStatus::Normal
    }
}

// ─── CLAP identity ────────────────────────────────────────────────────────────

impl ClapPlugin for SynesthesiaPlugin {
    const CLAP_ID:          &'static str          = "com.synesthesia.plugin";
    const CLAP_DESCRIPTION: Option<&'static str>  = Some("Synesthesia ML Music Plugin");
    const CLAP_MANUAL_URL:  Option<&'static str>  = None;
    const CLAP_SUPPORT_URL: Option<&'static str>  = None;

    // [HUMAN DECISION 3] — affects DAW browser categorization
    const CLAP_FEATURES: &'static [ClapFeature] = &[
        ClapFeature::Instrument,
        ClapFeature::Synthesizer,
        ClapFeature::Stereo,
    ];
}

// ─── VST3 identity ────────────────────────────────────────────────────────────

impl Vst3Plugin for SynesthesiaPlugin {
    // [HUMAN DECISION 1] — 16 bytes, globally unique, NEVER change after first DAW session saved
    const VST3_CLASS_ID: [u8; 16] = *b"Synesthesia_Plug";

    const VST3_SUBCATEGORIES: &'static [Vst3SubCategory] = &[
        Vst3SubCategory::Instrument,
        Vst3SubCategory::Synth,
        Vst3SubCategory::Stereo,
    ];
}

// ─── Export macros ────────────────────────────────────────────────────────────
// These generate the C ABI entry points the DAW calls.

nih_export_clap!(SynesthesiaPlugin);
nih_export_vst3!(SynesthesiaPlugin);

xtask/Cargo.toml

[package]
name    = "xtask"
version = "0.1.0"
edition.workspace = true

[[bin]]
name = "xtask"
path = "src/main.rs"

[dependencies]
nih_plug_xtask = { workspace = true }

xtask/src/main.rs

fn main() {
    nih_plug_xtask::main()
}

---

Building and Installing the Plugin

Build the plugin bundle

# From workspace root — bundles both VST3 and CLAP
cargo xtask bundle synesthesia-plugin --release

Output locations:

target/bundled/
├── Synesthesia.vst3/
│   └── Contents/
│       └── x86_64-win/
│           └── Synesthesia.vst3     ← the DLL (despite the .vst3 extension)
└── Synesthesia.clap                 ← single file for CLAP hosts

Install for testing in Reaper

# VST3 — copy entire .vst3 folder (not just the DLL inside)
cp -r target/bundled/Synesthesia.vst3 "C:/Program Files/Common Files/VST3/"

# CLAP
cp target/bundled/Synesthesia.clap "C:/Program Files/Common Files/CLAP/"

Restart Reaper and run Options → Preferences → Plug-ins → VST → Re-scan. The plugin should appear under Synesthesia / Ash.

Add to Taskfile

Add these targets to Taskfile.yml (T-007 owns the Taskfile, but register these targets there):

tasks:
  build:plugin:
    desc: "Build VST3 + CLAP plugin bundles"
    cmd: cargo xtask bundle synesthesia-plugin --release

  install:plugin:
    desc: "Install plugin bundles to system VST3/CLAP directories"
    cmds:
      - cp -r target/bundled/Synesthesia.vst3 "C:/Program Files/Common Files/VST3/"
      - cp target/bundled/Synesthesia.clap "C:/Program Files/Common Files/CLAP/"

---

Implementation Notes

Why cdylib and not rlib

cdylib produces a C-ABI dynamic library (.dll on Windows). The DAW loads this with LoadLibrary and calls the VST3/CLAP entry points that nih_export_vst3! and nih_export_clap! generate. rlib is a Rust-only library format — the DAW cannot load it.

synesthesia-core link check

use synesthesia_core as _core; in lib.rs compiles but does nothing. Its purpose is to fail the build if synesthesia-core becomes unreachable or changes its public API in a breaking way. This is intentional dead-code usage — suppress with #[allow(unused_imports)] if clippy warns.

Buffer size in DAW context

Unlike T-004's fixed 512-sample buffer, the DAW calls process() with whatever buffer size the user configured — commonly 256, 512, or 1024. nih-plug's Buffer abstraction handles this; T-005's passthrough loop works correctly for any size. When T-032 adds Perf RNN generation, it will need to handle variable buffer sizes — that's T-032's concern.

VST3_CLASS_ID collision risk

Steinberg maintains no central registry for VST3 class IDs. The risk of collision with another plugin is real but low — there are tens of thousands of VST3 plugins. Using *b"Synesthesia_Plug" (human-readable ASCII) is less likely to collide than a random GUID would be with a generic name. If you want a guaranteed unique ID, use a UUID generator and convert to [u8; 16]. That is [HUMAN DECISION 1].

MIDI_INPUT: MidiConfig::Basic

Setting Basic means the DAW routes note events to this plugin even though T-005 doesn't process them. This prevents DAWs from graying out MIDI routing to this plugin in their mixer. Setting it to None here and changing to Basic in T-032 would require DAW rescan/session reload.

---

Acceptance Criteria

• cargo check --workspace — zero errors including plugin/ and xtask/
• cargo clippy --workspace -- -D warnings — zero warnings
• cargo xtask bundle synesthesia-plugin --release completes without error
• target/bundled/Synesthesia.vst3/ directory exists with DLL inside
• target/bundled/Synesthesia.clap file exists
• Plugin loads in Reaper without crash (no red error in plugin scanner)
• Plugin appears in Reaper as Synesthesia under vendor Ash (or updated [HUMAN DECISION 2] value)
• Audio passthrough works: audio routed through the plugin passes without silence, clipping, or crash
• Gain knob visible in Reaper plugin GUI (nih-plug provides generic UI for free)
• PR comment with [HUMAN DECISION] section posted before marking ready for review
• Human has replied confirming or updating the three identity constants

---

Testing

Automated

• cargo check --workspace — zero errors
• cargo clippy --workspace -- -D warnings — zero warnings

Manual

1. Build succeeds

cargo xtask bundle synesthesia-plugin --release
ls target/bundled/
# Expected: Synesthesia.vst3/ and Synesthesia.clap

2. Reaper loads plugin

• Copy target/bundled/Synesthesia.vst3/ to VST3 system folder
• Reaper: Options → Preferences → VST → Re-scan
• Add an FX track, search "Synesthesia" — should appear
• Open plugin: generic nih-plug UI shows "Gain" knob at 0.0 dB

3. Audio passthrough

• Route an audio track through the plugin
• Play audio — output should be identical to input at 0 dB gain
• Move gain knob to –∞ — silence
• Move gain knob to +6 dB — louder (clip if input is loud)

4. No xrun on buffer-size mismatch

• Set Reaper buffer size to 256, play audio through plugin — no crackling
• Set buffer size to 1024 — still no crackling

5. Standalone + plugin both build

cargo build --package kansas --release
cargo xtask bundle synesthesia-plugin --release
# Both should succeed without conflict

---

Iteration Protocol

## Blocker or Human Decision — [date]
Decision/criterion: [verbatim]
Current value: [what was set]
Issue: [what's wrong or needs confirmation]
Recommendation: [agent's suggestion]

Set status:roadmap if blocked. Post the [HUMAN DECISION] PR comment before requesting review.

---

GitHub CLI

gh issue create \
  --title "T-005: nih-plug VST3/CLAP scaffold — audio passthrough, shared core, bundler" \
  --label "type:task,stack:rust,agent:human-led,priority:critical,status:ready,day:1" \
  --body-file T-005.md

---

Parent: GENESIS
Blocks: T-032 (Perf RNN VST parameters), T-065 (Perf RNN standalone), T-069 (Perf RNN MIDI output VST), T-089 (Magenta RT VST audio), T-103 (DDSP + Steam Audio in VST), T-143 (pluginval CI)
Blocked By: T-001
Version: v0.1 · Iteration: iter-1 · Effort: M (1–2 days)