--- license: mit base_model: microsoft/deberta-v3-base language: - en library_name: onnx tags: - token-classification - intent-classification - onnx - daw - music-production --- # MAGDA command model Turns a typed request into a [MAGDA](https://github.com/Conceptual-Machines/magda-core) DAW command, offline and on-device. "put @serum on a new track" becomes `track(name="", new=true).fx.add(name="")`, which the DAW executes as one undoable transaction. Fine-tuned from `microsoft/deberta-v3-base`. English only. ## Scope: edits, not gestures The command set covers **editing operations** — creating and naming tracks, adding devices, building racks, clip and note operations, grouping, selection by predicate. Deliberately excluded are real-time transport and mixer states (play/stop, track mute, solo): those have keyboard shortcuts, and a model with latency is strictly worse than the button. Queries ("what's the tempo") are excluded too — they are answers, not edits. Those requests are not merely unsupported; they are trained as an explicit **abstain** class, so the model returns nothing rather than executing whichever command happened to be nearest. A closed label set cannot decline unless declining is itself a label. Note that clip enable/disable *is* covered: it is a persistent property of the project, unlike a mixer mute, and picking one clip out by name is exactly the kind of thing typing beats clicking at. ## What it actually does **It never writes the command.** It does perception only, in a single forward pass, and emits two things: 1. which of 39 commands this is (one of which is "none of these") 2. a BIO tag per word saying what role that word plays ``` "can you drop @1176 onto the vocals" O O O PLUGIN O O TRACK_NAME intent = add_plugin ``` Deterministic code then assembles the DSL from those labels. The model cannot emit malformed output, because it does not emit output — a property a generative model would have to buy back with constrained decoding. Plugin references never reach the model as identities: the host rewrites every `@mention` to an opaque `` token before tokenization and substitutes the real one back by position. So an unseen plugin generalizes for free, and the vocabulary does not grow with the plugin registry. ## Results Scored as exact-match on the fully rendered command, not on tags. | | in-distribution | **held-out phrasing** | |---|---|---| | previous model (51k-param conv net, from scratch) | 98.2% | **46.5%** | | **this model** | 100% | **~90%** | The two columns matter more than the numbers. The in-distribution set is authored from the same case list the training templates come from, so a high score there means the model recalls its templates. The held-out set is 68 cases hand-written from the command list alone, in the register people actually type: lowercase, filler, contractions, typos. That is the number that predicts behaviour on input nobody has seen. The previous model had no language prior — trained from scratch on synthetic English, it named tracks after whichever word sat in the name position during training: ``` "can you drop @1176 onto the vocals" -> track(name="Can You Drop @1176")... "push Keys up to 2 dB" -> track(name="Push").track.set(volume_db=2) "the bass needs @pro_q_3" -> track(name="Bass Needs").track.set(colour="None") ``` Politeness wrappers scored 10%. With a pretrained encoder they score 100%. ## Files | file | size | what | |---|---|---| | `command_model.onnx` | 442 MB | encoder + intent head + slot head, one graph, opset 17 | | `tokenizer.json` | 8 MB | HF fast tokenizer, including the `` special tokens | | `maps.json` | 1 KB | intent and BIO tag id maps | Inputs are `input_ids` and `attention_mask`, both `int64 [batch, seq]` with dynamic sequence length. Outputs are `intent_logits [batch, 39]` and `slot_logits [batch, seq, 20]`. ## Quantization warning The embedding is int8; **the attention and FFN deliberately are not.** | build | size | accuracy | |---|---|---| | fp32 | 736 MB | 96.5% | | int8 embedding only (this) | 442 MB | 96.1% | | int8 everything | 538 MB | **0.9%** | DeBERTa-v3's disentangled attention is quantization-hostile. Running `quantize_dynamic` over the whole graph produces a model that loads, runs at full speed, and emits garbage. If you re-quantize this, re-measure it. ## Usage Two details will cost you accuracy if you get them wrong. **Words, not subwords.** The tags are per word, and the deterministic reconstructor consumes per-word tags. Label the first subword of each word and read the tag back at that same index. Reading tags positionally shifts every tag after the first multi-piece word — it still produces plausible-looking output, so it fails quietly. **The alias tokens must stay atomic.** `` and `` are registered as additional special tokens. If your tokenizer splits them into `<`, `alias`, `>`, every plugin slot breaks. ```python from transformers import AutoTokenizer import onnxruntime as ort, numpy as np tok = AutoTokenizer.from_pretrained("microsoft/deberta-v3-base", from_slow=True) tok.add_special_tokens({"additional_special_tokens": ["", ""]}) sess = ort.InferenceSession("command_model.onnx") words = ["put", "", "on", "a", "new", "track"] enc = tok(words, is_split_into_words=True, return_tensors="np") intent_logits, slot_logits = sess.run(None, { "input_ids": enc["input_ids"].astype(np.int64), "attention_mask": enc["attention_mask"].astype(np.int64), }) ``` A reference implementation in C++ (ONNX Runtime plus a hand-written SentencePiece Unigram tokenizer) is in `magda/agents/command_model_onnx.cpp` and `unigram_tokenizer.cpp` in the MAGDA repo. ## Training 24,000 template-generated English examples (no LLM paraphrasing in this build), 4 epochs, lr 3e-5 for the encoder and 1e-3 for the two heads. Checkpoints are selected on a hand-authored held-out set, never on the reported one — the val split is drawn from the same templates as training and saturates at 100% after one epoch, which makes it useless for ranking checkpoints. ## Limitations - **English only.** The word splitter in front of the model is ASCII-only, so non-English input reaches it as an empty sequence. This is a limitation of the surrounding pipeline, not the encoder, which is multilingual-capable. - **Plugin references need the `@` sigil.** `track with @serum` works; `track with serum` does not, and degrades into an unrelated command rather than failing cleanly. - **Closed command set.** 38 commands plus an explicit abstain class. Anything outside them returns nothing rather than being mapped onto the nearest command — but abstain recall is not perfect, so an unusual out-of-scope request can still land on a command. - Sentences where the verb trails its object ("i don't need the perc track anymore, delete it") are a known weak spot. ## License MIT, following `microsoft/deberta-v3-base`.