github-issues/T-005.md

# T-005: nih-plug VST3/CLAP Scaffold — Audio Passthrough, Shared Core Crate, Bundler

**Type:** Task  
**Phase:** 0 — Foundation  
**Autonomy:** `agent:human-led` (A3) — Agent implements the full scaffold. Human must confirm the three identity constants marked `[HUMAN DECISION]` before the PR is merged. Everything else is fully specified.  
**Stack:** `stack:rust`  
**Version:** v0.1  
**Iteration:** iter-1  
**Effort:** M (1–2 days)

---

> ⚠️ **Agent Scope:** Create the `plugin/` crate with nih-plug, implement stereo audio passthrough, set up the `xtask` bundler, and verify the plugin loads in Reaper. Do **not** add any model inference, IPC calls, or generation logic — those arrive in T-032 (Perf RNN VST), T-065, T-089 (Magenta RT VST). Parameters beyond a single placeholder `gain` are also out of scope.

---

## Context

Synesthesia ships as both a standalone Tauri desktop app and a VST3/CLAP plugin. Both compile from the same `synesthesia-core` library crate. The plugin uses `nih-plug` — MIT-licensed, Rust-native, produces VST3 + CLAP from the same source, no JUCE commercial license required.

**Why not JUCE:** JUCE's GPL licence requires open-sourcing your plugin or purchasing a commercial licence. nih-plug is MIT, ships as a single Rust crate, and the community has validated it against Reaper, Ableton, Bitwig, and FL Studio.

**Dual-target build model:**

```
synesthesia-core   (library crate)
      ├── kansas/           ← Tauri app depends on this
      └── plugin/           ← nih-plug plugin depends on this
```

`synesthesia-core` is the only place shared business logic lives. Neither `kansas/` nor `plugin/` duplicates code. T-006 scaffolds `synesthesia-core`'s content; T-005 just adds `plugin/` as a consumer of it.

---

## Human Decisions Required Before Merge

The following three constants in `plugin/src/lib.rs` must be reviewed and confirmed by the human before the PR is approved. The values below are **agent defaults** — the human changes them if desired:

```rust
// [HUMAN DECISION 1] — VST3 Class ID
// Must be exactly 16 bytes and globally unique per plugin.
// This ID is baked into every DAW project that uses the plugin.
// Changing it after release breaks saved sessions.
// Default (agent sets this): ASCII-padded plugin name
const VST3_CLASS_ID: [u8; 16] = *b"Synesthesia_Plug";

// [HUMAN DECISION 2] — Plugin metadata shown in DAW
const VENDOR: &str = "Ash";
const URL: &str = "https://github.com/ashiedu/synesthesia";

// [HUMAN DECISION 3] — CLAP feature tags (affects DAW browser categorization)
const CLAP_FEATURES: &[ClapFeature] = &[
    ClapFeature::Instrument,
    ClapFeature::Synthesizer,
    ClapFeature::Stereo,
];
```

Open a PR comment with `[HUMAN DECISION]` in the title containing the proposed values. Do not merge until the human replies with explicit confirmation or updated values.

---

## Prerequisites

- [ ] T-001 merged — workspace `Cargo.toml` with `edition = "2024"`, `crates/synesthesia-core/` exists as empty lib
- [ ] `cargo check --workspace` passes before starting

> Note: T-004 (CPAL audio) and T-006 (`synesthesia-core` scaffold) do not need to be merged first. T-005 only requires the workspace to exist.

---

## New Dependencies

nih-plug is **not on crates.io** — it is a git dependency. The wildcard version policy does not apply to git deps; use the git URL directly.

Add to root `Cargo.toml` `[workspace.dependencies]`:

```toml
# nih-plug — git only, not on crates.io
nih_plug      = { git = "https://github.com/robbert-vdh/nih-plug.git", features = ["vst3", "clap"] }
nih_plug_xtask = { git = "https://github.com/robbert-vdh/nih-plug.git" }
```

> `Cargo.lock` will pin these to the HEAD commit at first `cargo build`. Commit `Cargo.lock` so the pinned revision is reproducible.

---

## Workspace Changes (`Cargo.toml`)

```toml
[workspace]
members = [
    "kansas",
    "crates/*",
    "plugin",
    "xtask",        # cargo xtask bundler
]
resolver = "2"
```

---

## File Structure After This Task

```
synesthesia/
├── Cargo.toml               ← updated: adds plugin, xtask members
├── plugin/
│   ├── Cargo.toml
│   └── src/
│       ├── lib.rs           ← Plugin impl (passthrough + params stub)
│       └── params.rs        ← SynesthesiaParams (single gain knob)
└── xtask/
    ├── Cargo.toml
    └── src/
        └── main.rs          ← delegates to nih_plug_xtask::main()
```

---

## Implementation

### `plugin/Cargo.toml`

```toml
[package]
name    = "synesthesia-plugin"
version = "0.1.0"
edition.workspace = true

# REQUIRED: cdylib produces the .dll/.so/.dylib the DAW loads
[lib]
name      = "synesthesia_plugin"
crate-type = ["cdylib"]

[dependencies]
nih_plug         = { workspace = true }
synesthesia-core = { path = "../crates/synesthesia-core" }
```

### `plugin/src/params.rs`

```rust
use nih_plug::prelude::*;

/// Plugin parameters.
/// T-005: single placeholder gain knob — proves the parameter system compiles.
/// T-032 (Perf RNN) adds: Temperature, Primer, BPM Sync
/// T-065 (Perf RNN standalone) adds: channel routing
/// T-089 (Magenta RT) adds: Style Blend 1–6
#[derive(Params)]
pub struct SynesthesiaParams {
    /// Output gain in dB. Range –96 to +6. Default 0 dB.
    /// This is a smoke-test parameter; it is not used in T-005's passthrough.
    #[id = "gain"]
    pub gain: FloatParam,
}

impl Default for SynesthesiaParams {
    fn default() -> Self {
        Self {
            gain: FloatParam::new(
                "Gain",
                util::db_to_gain(0.0),
                FloatRange::Skewed {
                    min: util::db_to_gain(-96.0),
                    max: util::db_to_gain(6.0),
                    factor: FloatRange::gain_skew_factor(-96.0, 6.0),
                },
            )
            .with_smoother(SmoothingStyle::Logarithmic(50.0))
            .with_unit(" dB")
            .with_value_to_string(formatters::v2s_f32_gain_to_db(2))
            .with_string_to_value(formatters::s2v_f32_gain_to_db()),
        }
    }
}
```

### `plugin/src/lib.rs`

```rust
use std::sync::Arc;
use nih_plug::prelude::*;

mod params;
use params::SynesthesiaParams;

// Unused in T-005; imported to verify synesthesia-core links correctly.
// T-006 populates this crate; for now it's just a link-time check.
use synesthesia_core as _core;

// ─── Plugin struct ────────────────────────────────────────────────────────────

pub struct SynesthesiaPlugin {
    params: Arc<SynesthesiaParams>,
    sample_rate: f32,
}

impl Default for SynesthesiaPlugin {
    fn default() -> Self {
        Self {
            params: Arc::new(SynesthesiaParams::default()),
            sample_rate: 44_100.0,
        }
    }
}

// ─── nih-plug Plugin trait ────────────────────────────────────────────────────

impl Plugin for SynesthesiaPlugin {
    // ── Identity ──────────────────────────────────────────────────────────────

    const NAME:    &'static str = "Synesthesia";
    const VENDOR:  &'static str = "Ash";            // [HUMAN DECISION 2]
    const URL:     &'static str = "https://github.com/ashiedu/synesthesia"; // [HUMAN DECISION 2]
    const EMAIL:   &'static str = "";
    const VERSION: &'static str = env!("CARGO_PKG_VERSION");

    // ── Audio I/O ─────────────────────────────────────────────────────────────

    /// Stereo in, stereo out. The DAW may also call us with no inputs (instrument mode).
    const AUDIO_IO_LAYOUTS: &'static [AudioIOLayout] = &[
        AudioIOLayout {
            main_input_channels:  NonZeroU32::new(2),
            main_output_channels: NonZeroU32::new(2),
            aux_input_ports:  &[],
            aux_output_ports: &[],
            names: PortNames::const_default(),
        },
        // Instrument mode — no audio input, stereo output
        AudioIOLayout {
            main_input_channels:  None,
            main_output_channels: NonZeroU32::new(2),
            aux_input_ports:  &[],
            aux_output_ports: &[],
            names: PortNames::const_default(),
        },
    ];

    // ── MIDI ──────────────────────────────────────────────────────────────────
    // T-068 (MIDI output) activates BasicMidi output.
    // T-005: accept MIDI so DAWs route notes to us; output is off until T-068.
    const MIDI_INPUT:  MidiConfig = MidiConfig::Basic;
    const MIDI_OUTPUT: MidiConfig = MidiConfig::None;

    type SysExMessage  = ();
    type BackgroundTask = ();

    // ── Parameters ────────────────────────────────────────────────────────────

    fn params(&self) -> Arc<dyn Params> {
        self.params.clone()
    }

    // ── Lifecycle ─────────────────────────────────────────────────────────────

    fn initialize(
        &mut self,
        _audio_io_layout: &AudioIOLayout,
        buffer_config: &BufferConfig,
        _context: &mut impl InitContext<Self>,
    ) -> bool {
        self.sample_rate = buffer_config.sample_rate;
        // T-046+: initialize ONNX runtime here when model inference is added
        true
    }

    fn reset(&mut self) {
        // T-062+: reset generator state (e.g., Perf RNN LSTM hidden state) here
    }

    // ── Audio processing ──────────────────────────────────────────────────────

    fn process(
        &mut self,
        buffer: &mut Buffer,
        _aux: &mut AuxiliaryBuffers,
        _context: &mut impl ProcessContext<Self>,
    ) -> ProcessStatus {
        // T-005: passthrough with gain applied.
        // T-032 replaces this with Perf RNN MIDI generation.
        // T-065 adds Magenta RT audio generation.
        // T-095 adds DDSP timbre transfer.

        let gain = self.params.gain.smoothed.next();

        for channel_samples in buffer.iter_samples() {
            for sample in channel_samples {
                *sample *= gain;
            }
        }

        ProcessStatus::Normal
    }
}

// ─── CLAP identity ────────────────────────────────────────────────────────────

impl ClapPlugin for SynesthesiaPlugin {
    const CLAP_ID:          &'static str          = "com.synesthesia.plugin";
    const CLAP_DESCRIPTION: Option<&'static str>  = Some("Synesthesia ML Music Plugin");
    const CLAP_MANUAL_URL:  Option<&'static str>  = None;
    const CLAP_SUPPORT_URL: Option<&'static str>  = None;

    // [HUMAN DECISION 3] — affects DAW browser categorization
    const CLAP_FEATURES: &'static [ClapFeature] = &[
        ClapFeature::Instrument,
        ClapFeature::Synthesizer,
        ClapFeature::Stereo,
    ];
}

// ─── VST3 identity ────────────────────────────────────────────────────────────

impl Vst3Plugin for SynesthesiaPlugin {
    // [HUMAN DECISION 1] — 16 bytes, globally unique, NEVER change after first DAW session saved
    const VST3_CLASS_ID: [u8; 16] = *b"Synesthesia_Plug";

    const VST3_SUBCATEGORIES: &'static [Vst3SubCategory] = &[
        Vst3SubCategory::Instrument,
        Vst3SubCategory::Synth,
        Vst3SubCategory::Stereo,
    ];
}

// ─── Export macros ────────────────────────────────────────────────────────────
// These generate the C ABI entry points the DAW calls.

nih_export_clap!(SynesthesiaPlugin);
nih_export_vst3!(SynesthesiaPlugin);
```

### `xtask/Cargo.toml`

```toml
[package]
name    = "xtask"
version = "0.1.0"
edition.workspace = true

[[bin]]
name = "xtask"
path = "src/main.rs"

[dependencies]
nih_plug_xtask = { workspace = true }
```

### `xtask/src/main.rs`

```rust
fn main() {
    nih_plug_xtask::main()
}
```

---

## Building and Installing the Plugin

### Build the plugin bundle

```bash
# From workspace root — bundles both VST3 and CLAP
cargo xtask bundle synesthesia-plugin --release
```

Output locations:
```
target/bundled/
├── Synesthesia.vst3/
│   └── Contents/
│       └── x86_64-win/
│           └── Synesthesia.vst3     ← the DLL (despite the .vst3 extension)
└── Synesthesia.clap                 ← single file for CLAP hosts
```

### Install for testing in Reaper

```bash
# VST3 — copy entire .vst3 folder (not just the DLL inside)
cp -r target/bundled/Synesthesia.vst3 "C:/Program Files/Common Files/VST3/"

# CLAP
cp target/bundled/Synesthesia.clap "C:/Program Files/Common Files/CLAP/"
```

Restart Reaper and run **Options → Preferences → Plug-ins → VST → Re-scan**. The plugin should appear under `Synesthesia / Ash`.

### Add to Taskfile

Add these targets to `Taskfile.yml` (T-007 owns the Taskfile, but register these targets there):

```yaml
tasks:
  build:plugin:
    desc: "Build VST3 + CLAP plugin bundles"
    cmd: cargo xtask bundle synesthesia-plugin --release

  install:plugin:
    desc: "Install plugin bundles to system VST3/CLAP directories"
    cmds:
      - cp -r target/bundled/Synesthesia.vst3 "C:/Program Files/Common Files/VST3/"
      - cp target/bundled/Synesthesia.clap "C:/Program Files/Common Files/CLAP/"
```

---

## Implementation Notes

### Why `cdylib` and not `rlib`

`cdylib` produces a C-ABI dynamic library (`.dll` on Windows). The DAW loads this with `LoadLibrary` and calls the VST3/CLAP entry points that `nih_export_vst3!` and `nih_export_clap!` generate. `rlib` is a Rust-only library format — the DAW cannot load it.

### `synesthesia-core` link check

`use synesthesia_core as _core;` in `lib.rs` compiles but does nothing. Its purpose is to fail the build if `synesthesia-core` becomes unreachable or changes its public API in a breaking way. This is intentional dead-code usage — suppress with `#[allow(unused_imports)]` if clippy warns.

### Buffer size in DAW context

Unlike T-004's fixed 512-sample buffer, the DAW calls `process()` with whatever buffer size the user configured — commonly 256, 512, or 1024. nih-plug's `Buffer` abstraction handles this; T-005's passthrough loop works correctly for any size. When T-032 adds Perf RNN generation, it will need to handle variable buffer sizes — that's T-032's concern.

### VST3_CLASS_ID collision risk

Steinberg maintains no central registry for VST3 class IDs. The risk of collision with another plugin is real but low — there are tens of thousands of VST3 plugins. Using `*b"Synesthesia_Plug"` (human-readable ASCII) is less likely to collide than a random GUID would be with a generic name. If you want a guaranteed unique ID, use a UUID generator and convert to `[u8; 16]`. That is [HUMAN DECISION 1].

### `MIDI_INPUT: MidiConfig::Basic`

Setting `Basic` means the DAW routes note events to this plugin even though T-005 doesn't process them. This prevents DAWs from graying out MIDI routing to this plugin in their mixer. Setting it to `None` here and changing to `Basic` in T-032 would require DAW rescan/session reload.

---

## Acceptance Criteria

- [ ] `cargo check --workspace` — zero errors including `plugin/` and `xtask/`
- [ ] `cargo clippy --workspace -- -D warnings` — zero warnings
- [ ] `cargo xtask bundle synesthesia-plugin --release` completes without error
- [ ] `target/bundled/Synesthesia.vst3/` directory exists with DLL inside
- [ ] `target/bundled/Synesthesia.clap` file exists
- [ ] Plugin loads in Reaper without crash (no red error in plugin scanner)
- [ ] Plugin appears in Reaper as `Synesthesia` under vendor `Ash` (or updated [HUMAN DECISION 2] value)
- [ ] Audio passthrough works: audio routed through the plugin passes without silence, clipping, or crash
- [ ] Gain knob visible in Reaper plugin GUI (nih-plug provides generic UI for free)
- [ ] PR comment with `[HUMAN DECISION]` section posted before marking ready for review
- [ ] Human has replied confirming or updating the three identity constants

---

## Testing

### Automated
- [ ] `cargo check --workspace` — zero errors
- [ ] `cargo clippy --workspace -- -D warnings` — zero warnings

### Manual

**1. Build succeeds**
```bash
cargo xtask bundle synesthesia-plugin --release
ls target/bundled/
# Expected: Synesthesia.vst3/ and Synesthesia.clap
```

**2. Reaper loads plugin**
- Copy `target/bundled/Synesthesia.vst3/` to VST3 system folder
- Reaper: Options → Preferences → VST → Re-scan
- Add an FX track, search "Synesthesia" — should appear
- Open plugin: generic nih-plug UI shows "Gain" knob at 0.0 dB

**3. Audio passthrough**
- Route an audio track through the plugin
- Play audio — output should be identical to input at 0 dB gain
- Move gain knob to –∞ — silence
- Move gain knob to +6 dB — louder (clip if input is loud)

**4. No xrun on buffer-size mismatch**
- Set Reaper buffer size to 256, play audio through plugin — no crackling
- Set buffer size to 1024 — still no crackling

**5. Standalone + plugin both build**
```bash
cargo build --package kansas --release
cargo xtask bundle synesthesia-plugin --release
# Both should succeed without conflict
```

---

## Iteration Protocol

```markdown
## Blocker or Human Decision — [date]
Decision/criterion: [verbatim]
Current value: [what was set]
Issue: [what's wrong or needs confirmation]
Recommendation: [agent's suggestion]
```

Set `status:roadmap` if blocked. Post the `[HUMAN DECISION]` PR comment before requesting review.

---

## GitHub CLI

```bash
gh issue create \
  --title "T-005: nih-plug VST3/CLAP scaffold — audio passthrough, shared core, bundler" \
  --label "type:task,stack:rust,agent:human-led,priority:critical,status:ready,day:1" \
  --body-file T-005.md
```

---

**Parent:** GENESIS  
**Blocks:** T-032 (Perf RNN VST parameters), T-065 (Perf RNN standalone), T-069 (Perf RNN MIDI output VST), T-089 (Magenta RT VST audio), T-103 (DDSP + Steam Audio in VST), T-143 (pluginval CI)  
**Blocked By:** T-001  
**Version:** v0.1 · **Iteration:** iter-1 · **Effort:** M (1–2 days)