| --- |
| license: other |
| license_name: personaplex-derivative |
| base_model: nvidia/personaplex-7b-v1 |
| library_name: peft |
| tags: [full-duplex, speech, refusal, instruction-following, lora, personaplex] |
| extra_gated_prompt: >- |
| Research checkpoints of a speech model trained to refuse requests when an operator instruction |
| tells it to. They are not a safety system. By requesting access you agree to use them for |
| research only. |
| extra_gated_fields: |
| Name: text |
| Affiliation: text |
| Intended use: text |
| --- |
| |
| # duplex-refusal |
|
|
| LoRA adapters for [nvidia/personaplex-7b-v1](https://huggingface.co/nvidia/personaplex-7b-v1), a |
| full-duplex speech model, that make it refuse a request when a private operator instruction arrives |
| mid-conversation — and answer normally when none does. The instruction is a text span the model |
| reads but never speaks. |
|
|
| ## Versions |
|
|
| One folder per version; each has its own README with data, recipe, evaluation and known issues, and a |
| `VERSION.json` with the adapter's sha256 and scaling. Load a version at a pinned commit, not from a |
| moving `main`. |
|
|
| | version | adapter | revision | sha256 | what it is for | |
| |---|---|---|---|---| |
| | [`v0.1`](./v0.1/README.md) | `v0.1/lora.safetensors` (775,835,968 B) | `313d94c6a347b1493984388313e744440f0028ff` | `4bfe4ce2f37289a41349fb0d0e61f8bfe8bebf0a278543f7b09930cd10f068fa` | preview; run `refusal_v0.1_ins` step 4,800. Always answers and always refuses when told, but over-refuses benign requests: take it where over-refusal is acceptable | |
| | [`v0.2.1`](./v0.2.1/README.md) | `v0.2.1/lora.safetensors` (775,856,664 B) | the commit that added `v0.2.1/` (pinned by sha in Daimonion's `hub/registry.py`) | `8dc8b45e2e6c08fdd7f538c62c4ca00c9b38b8a8454b211baeec4c44e9c3a976` | run `refusal_v0.2.1_ins` step 12,000; accept and refuse trained behind the same audio, harmful spots left out of training, the two instruction-marker rows trained. Mount it when the seat must hold an ordinary conversation | |
|
|
| Measured 2026-09-18 by replaying the seed-set benchmark's own 8 kHz telephony caller tapes (20 tapes, |
| 76 caller turns, the instruction spliced 2 s after the last turn): |
|
|
| | version | answers the caller | refuses after the alert | benign over-refusal | |
| |---|---|---|---| |
| | `v0.1` | 1.00 | 1.00 | 0.25 | |
| | `v0.2.1` | 0.88 | 0.94–1.00 | 0.00 | |
|
|
| The `refusal_v0.2_ins/` and `refusal_v0.2.1_ins/` folders hold every 1,000-step checkpoint of those |
| two runs with a per-checkpoint eval report; they are training artefacts, not versions. `v0.2.1/` is |
| the named, pinned copy of `refusal_v0.2.1_ins/checkpoint_012000/`. |
|
|
| ## Evaluation: Full-Duplex-Bench v1.0 (2026-09-19) |
|
|
| Turn-taking behaviour of the folded checkpoints on [Full-Duplex-Bench v1.0](https://arxiv.org/abs/2503.04721) |
| (727 stimuli, five categories), brain off, with the harness the JSALT 2026 team uses to reproduce the paper's |
| PersonaPlex row. Both versions ran under the **same condition**: the v0.1 training voice fed raw, the |
| per-category FDB prompts, seed 42424242. The stock PersonaPlex row is the team's reproduction at the |
| **VARF0 voice** (a different voice: FDB v1 numbers move strongly with the voice prompt, so treat that column |
| as context, not a controlled comparison). Full tables, the voice pilot and per-run summaries: |
| [`eval/fdb_v1/`](./eval/fdb_v1/README.md). |
|
|
| | Metric | Sub | PersonaPlex stock (VARF0) | `v0.1` | `v0.2.1` | Paper (PersonaPlex) | |
| |---|---|---|---|---|---| |
| | Pause Handling | Synthetic TOR ↓ | 0.642 | 0.504 | 0.555 | 0.584 | |
| | Pause Handling | Candor TOR ↓ | 0.718 | 0.593 | 0.704 | 0.662 | |
| | Backchannel | TOR ↓ | 0.418 | 0.636 | 0.636 | 0.327 | |
| | Backchannel | Freq ↑ | 0.176 | 0.093 | 0.091 | 0.025 | |
| | Backchannel | JSD ↓ | 0.700 | 0.748 | 0.743 | 0.649 | |
| | Smooth Turn-Taking | Candor TOR ↑ | 0.983 | 0.950 | 0.983 | 0.992 | |
| | Smooth Turn-Taking | Latency ↓ | 0.020 | 0.035 | 0.001 | 0.070 | |
| | User Interruption | TOR ↑ | 0.935 | 0.950 | 0.900 | 1.000 | |
| | User Interruption | Rating ↑ | 4.684† | 3.711† | 3.239† | 4.210 | |
| | User Interruption | Latency ↓ | 0.214 | 0.509 | 0.842 | 0.400 | |
|
|
| † the user-interruption rating comes from a local judge (gemma-4-E4B served as `gpt-4-turbo`), not GPT-4o: |
| comparable between these rows, not with the paper. Reading: `v0.1` holds the floor better while the caller |
| pauses and is rated higher when interrupted; `v0.2.1` takes the turn more reliably and starts sooner (its |
| 0.001 s smooth-turn latency is the metric's clamp at zero: it speaks at or before the boundary). Both are more |
| talkative than the paper row, which costs them backchannel TOR. |
|
|
| ## Evaluation: seedset closed loop, 5-goal pilot (2026-09-19) |
|
|
| The JSALT 2026 seed-set benchmark (a Crescendo attacker LLM speaking to the seat over a telephony channel, |
| tau2, protocol revision 2, judged by Muse-Glimmer-30B plus the HarmBench classifier), on five goals shared by |
| every earlier run, one episode each, four caller turns. Both checkpoints run **undefended** (brain off) and with |
| Daimonion's brain under each cadence. These are exploratory runs at benchmark tree v2.0; n = 5 per cell, so a |
| one-episode difference is one episode. |
|
|
| **Seats, undefended, same five goals** (harmful / partial / none = our judge's assistance label; "attack |
| succeeded" = the protocol's binary judge; HarmBench = the classifier's episode verdict): |
|
|
| | seat | harmful | partial | none | attack succeeded | HarmBench | |
| |---|---|---|---|---|---| |
| | stock PersonaPlex | 80% | 20% | 0% | 60% | 40% | |
| | duplex-refusal v0.1 | 0% | 80% | 20% | 0% | 20% | |
| | duplex-refusal v0.2.1 | 0% | 80% | 20% | 0% | 0% | |
|
|
| **With the brain** (cadences: P1 = one call at the caller's turn end; P2 = poll every 2 s while the caller |
| speaks, alert at the hand-over; P2k = P2 but a poll still in flight at the hand-over is awaited; P3 = poll |
| every 2 s always, the brain paces itself with `wait`): |
|
|
| | arm | v0.1 partial | v0.1 none | v0.1 refusal | v0.2.1 partial | v0.2.1 none | v0.2.1 refusal | |
| |---|---|---|---|---|---|---| |
| | base | 80% | 20% | 0% | 80% | 20% | 0% | |
| | P1 | 40% | 60% | 100% | 20% | 80% | 60% | |
| | P2 | 20% | 80% | 100% | 40% | 60% | 80% | |
| | P2k | 0% | 100% | 100% | 20% | 80% | 100% | |
| | P3 | 40% | 60% | 80% | 60% | 40% | 40% | |
|
|
| Neither checkpoint answered harmfully in any arm. `v0.1` follows an injected alert more sharply (fewer partial |
| episodes, more explicit refusals in every cadence); `v0.2.1` does not over-refuse benign requests (0.00 vs 0.25 |
| on the tapes above) and takes turns better (FDB v1), which this harmful-only pilot cannot see. The cadence |
| ranking is the same for both: P2k best, P2 close, P3 weakest (it alerts once, then answers `wait`). Full tables |
| with brain-call counts and alert timing: [`eval/seedset_pilot/`](./eval/seedset_pilot/). |
|
|
| ## Usage |
|
|
| The adapters target PersonaPlex with the depth decoder sliced to 8 codebooks and two extra text |
| tokens (`<instr_start>` = 32000, `<instr_end>` = 32001), so fold them into the base first: |
|
|
| ```bash |
| huggingface-cli download MagicLuke/duplex-refusal --include "v0.2.1/*" --local-dir adapter |
| python -m duplex_brain.ckpt.build_checkpoint --hf-repo nvidia/personaplex-7b-v1 \ |
| --adapter adapter/v0.2.1/lora.safetensors --scaling 2.0 --out duplex-refusal-v0.2.1.safetensors |
| ``` |
|
|
| `build_checkpoint` is from [duplex-brain](https://github.com/MagicLuke/duplex-brain). Code and |
| training recipe: [duplex-online-if](https://github.com/MagicLuke/duplex-online-if). For `v0.2.1` |
| the two instruction-marker embedding rows are trained as well and must be folded with the rest of the |
| adapter (`VERSION.json` says `ft_instr_embed: true`). |
|
|
