Vital Audio JEPA V5
V5 extends the feature-focused V4 encoder with exact, single-control interventions rendered by Vital 1.6.4. Its main goal is to preserve acoustic state while making synthesizer-control effects accessible to a downstream preset decoder.
Input and corpus
- 22,050 Hz stereo PCM16, exactly 6.0 seconds
- four-second note plus two-second release tail
- no amplitude normalization
- 137,825 accepted renders from 7,394 parent presets
- 115,943 exact one-control interventions and 21,882 performance variants
- 40 controls spanning oscillators, filters, amplitude envelope, and effects
- parent-disjoint train, validation, and test splits
The dataset is stored under
interventions_v5_22050_stereo_6s/ in duckking032/vital. Every accepted
intervention records its exact control ID, before/after values, signed delta,
magnitude, and direction.
Model and objectives
- 36,519,146 parameters total; 22,322,666 trainable
- four learned global tokens plus residual global/patch pooling
- V4 initialization at one-tenth the learning rate for pretrained parameters
- frozen V4 teacher anchoring during training
- JEPA, acoustic descriptor, stereo/spatial, and acoustic-delta objectives
- exact control identity, sign, magnitude, effect, prototype, and contrastive objectives
- exact-control-balanced mutation/reference pair sampling
Training ran for 10 epochs / 20,440 steps in 2,451 seconds. Checkpoint 15,000 was selected by the held-out control, acoustic, and retrieval comparison.
Untouched parent-held-out results
| Metric | Result |
|---|---|
| Mean acoustic descriptor correlation | 0.8499 |
| Mean acoustic descriptor MAE | 0.0419 |
| Acoustic-delta cosine | 0.2536 |
| Built-in 40-way control accuracy | 15.52% |
| Built-in control-sign accuracy | 62.59% |
| Built-in control-effect cosine | 0.1996 |
| Coarse retrieval top-1 | 43.79% |
| Coarse retrieval top-5 | 62.20% |
| Coarse MRR@10 | 0.5157 |
Chance accuracy is 2.5% for control identity and 50% for direction.
Frozen representation probes
Probes use 2,000 training parents and fully disjoint validation/test parents; the JEPA remains frozen.
| Probe | Linear | MLP |
|---|---|---|
| Acoustic descriptor correlation | 0.8253 | 0.8408 |
| Acoustic-delta correlation | 0.6003 | 0.6015 |
| 40-way control accuracy | 20.75% | 34.84% |
| 40-way balanced accuracy | 13.38% | 25.14% |
| Control-direction balanced accuracy | 70.70% | 72.94% |
The gap between the built-in control head and the frozen MLP probe shows that the encoder contains substantially more control information than the small training head extracts. This is encouraging for a learned preset decoder.
Limitations
Control information is uneven. Envelope controls and several on/off effects
are much easier than oscillator-2/3 and subtle spectral/unison parameters.
Control-direction centroid consistency is still modest (mean cosine 0.117),
and the encoder does not itself produce a complete .vital preset.
checkpoints/jepa_v5/selected.pt is the optimizer-free selected checkpoint.
The full evaluation, checkpoint comparison, and probe report are included.