You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

These adapters are trained on Fisher (LDC) and Seamless Interaction. The corpora themselves are not redistributed here. By requesting access you agree not to attempt to reconstruct or redistribute the underlying corpora.

oif — online-instruction-following SFT checkpoints

LoRA adapters for nvidia/personaplex-7b-v1 that make the model follow instructions injected mid-conversation by a separate text LLM. A directive is spliced into the model's inner-monologue stream as extra timesteps (ins_token mode) with the audio held silent, so the model reads the directive and acts on its next turn without ever speaking it.

Project: MagicLuke/duplex-online-if · recipe of record: docs/two_stage_training.md · per-recipe lab notes: docs/lab/ · RL post-training of the same base: MagicLuke/oif-instok-rl

One subfolder per published checkpoint, <run_tag>/checkpoint_<step>/.

run	step	what it is	downstream eval
`stage1f_final_ins`	4000	Stage 1 only — Fisher + Seamless directive injection, never saw a goal card. The checkpoint stage 2 was initialised from, and the peak of the stage-1 conditioning probe	none
`stage2_ins_stage1f`	337	The shipped SFT checkpoint. 337 steps of conv-synth goal arcs continued from stage 1 via `lora_init_from`	IF-Bench v1.3 + FDB v1.0

Read that eval column before choosing. stage1f_final_ins/4000 is published as the controlled stage-1 baseline for "what did stage 2 add" — it has never been benchmarked and it cannot initiate speech from a cold prompt (see below). If you want a usable model, use step 337, or the RL checkpoints.

`stage2_ins_stage1f/checkpoint_000337`

The SFT checkpoint of record. lora_init_from stage 1's step 4000, then 337 steps on conv-synth goal arcs; instr_lr 0.0 (delimiter rows frozen), instr_dropout 0.0, lr 1e-6, bs 2 × 8 ranks, 240 s windows. Eval loss 2.097 → 1.597 (text 0.715 → 0.365, audio 1.382 → 1.233).

lora.safetensors is 2,328,712,528 bytes, sha256 6ccafd8707471d89620a2767bec1efe866b4527fb60461e451705730ab5ce4d8, folded at --scaling 2.0.

IF-Bench v1.3 — 200 tasks, judge `Qwen/Qwen3.8-27B`

examiner	topic 1–5	completion	either-side	order	seat-flip ↓	naturalness
stage2-337 + brain	3.79	49.1%	58.4%	9.5%	4.6%	4.05
stage2-337 brain off	2.19	14.6%	19.6%	0.0%	3.6%	4.54
`oif-instok-rl` gdpo-duplex v1200 + brain	4.80	85.8%	91.4%	47.5%	42.5%	3.81
PersonaPlex-7B base	2.19	7.6%	10.6%	0.0%	21.5%	4.45

This checkpoint's real strength is role stability, not completion. At 3.6–4.6% quote-verified seat flips it is an order of magnitude steadier than either RL checkpoint (40–66%) and better than the base model (21.5%), and its naturalness is the highest of our checkpoints. It completes far fewer tasks. Pick accordingly.

FDB v1.0 — 727 stimuli

condition	pause TOR syn/candor ↓	backchannel TOR/freq/JSD	smooth TOR/lat	interrupt TOR/rating/lat
brain off, training card + voice	0.124 / 0.236	0.073 / 0.011 / 0.953	0.857 / 0.992	0.960 / 2.312 / 0.908
brain off, VARF0 + FDB prompts	0.153 / 0.245	0.018 / 0.056 / 0.795	0.756 / 0.007	0.950 / 4.037 / 0.859
raw adapter, NATF2 + FDB prompts	0.000 / 0.009	0.255 / 0.045 / 0.786	0.244 / 0.997	0.930 / 3.914 / 0.977
PersonaPlex (ours, VARF0)	0.642 / 0.718	0.418 / 0.176 / 0.700	0.983 / 0.020	0.935 / 4.684 / 0.214

The finetune regressed turn-taking on this benchmark — smooth-turn TOR fell 0.96 → 0.24 in the raw-adapter condition. Low pause TOR is good (it holds silence), but taken together with the smooth-turn collapse the picture is a model that has become reluctant to take the floor.

Training-data overlap with IF-Bench — measured

Disclosure: conv-synth v1 was seeded from the 200 FDB-v2 staged tasks and split by scenario into 180 train / 10 val / 10 test — and those 200 scenarios are the 200 IF-Bench task ids. So stage 2 trained on scenarios matching 180 of the 200 benchmark tasks. (Stage 1 did not; its corpus is Fisher + Seamless.)

Verified completion on the 20 genuinely held-out scenarios minus the 180 trained ones, bootstrap CI over tasks:

arm	held-out (n=19)	trained (n=172)	gap	95% CI
stage2-337 brain off	18.4%	14.8%	+3.6	[−5.7, +14.3]
stage2-337 brain on	50.0%	49.9%	+0.1	[−13.0, +13.4]

No memorization advantage is detectable — both CIs span zero and the point estimates are flat-to-positive. The overlap is real in provenance but does not show up in the score. The test has limited power at n≈20 (roughly ±10 points), so it rules out a large effect, not a small one. Held-out ids: fdb-v2-IF-bench/benchmark/oif_splits/convsynth_overlap.json; score both halves with slurm/arm_compare.py --subsets.

`stage1f_final_ins/checkpoint_004000`

Stage 1 of the two-stage recipe, stopped for publication at the step stage 2 branched from.

Base personaplex-7b-v1-native-dq8-instr32002 — dep_q sliced 16→8, text vocab 32000→32002 for the two directive markers. The adapter does not load onto stock PersonaPlex unchanged; see the repo's scripts/convert/.
Data instr_full/v2 — Fisher-train + Seamless improvised + Seamless naturalistic, weighted, 240 s windows. 221,247 examples / 1,652,922 injections.
Recipe LoRA r512 + depth_rank 64, scaling 2.0 · lr 2e-6, instr_lr 2e-5 · instr_dropout 0.5→0.3 · instr_near_p 0.7 / near_frames 25 · text_padding_weight 0.3 · first_codebook_weight_multiplier 100 · bs 8.

Measured behaviour, including what is wrong with it

Directive probe on 200 held-out Fisher dialogues (1,850 scored injections), wrong − correct cross-entropy, base-corrected:

checkpoint	base	1000	2000	3000	4000	5000	6000	7000
base-corrected Δ	—	+0.088	+0.085	+0.088	+0.100	+0.091	+0.092	+0.081

Steering saturates at the first saved checkpoint. Steps 1000 → 7000 move the conditioning delta by less than 0.02, and the peak is at 4000, not at the end.

This checkpoint cannot initiate speech from a cold prompt. Free-running from a voice+persona prompt with a real partner talking and no teacher-forced history, it produces 0 words in 120 s where the base model produces 41. The same checkpoint looks better than base under any fixture that supplies 30 s of real conversation first (conditionality delta +0.257 vs base's +0.116) — it can continue speech but not start it. Treat teacher-forced numbers for this checkpoint as measuring conditioning only, never conversational competence.

It is published because it is the controlled stage-1 baseline for "what did stage 2 add", not because it is a usable conversational model on its own.

Why ins_token. Every variant that routes the instruction through a trainable projection measured at chance, including on this exact corpus: add_sum reached Δ = +0.0002 ± 0.0003 after 4000 steps — correct, wrong, and no instruction at all were identical to three decimals, while its eval loss fell normally. Only mechanisms carrying real token ids (inline, ins_token) ever learned. That is the single most useful negative result here.

Usage

Instructions ride as real text-token spans spliced in as extra timesteps:

span = [32000, *spm.encode("ask them about their job"), 32001]   # <instr_start> … <instr_end>

Fold into NVIDIA's base with the tested recipe from duplex-brain:

huggingface-cli download MagicLuke/oif-instok-sft \
    --include "stage2_ins_stage1f/checkpoint_000337/*" --local-dir adapter
python -m duplex_brain.ckpt.build_checkpoint \
    --hf-repo nvidia/personaplex-7b-v1 \
    --adapter adapter/stage2_ins_stage1f/checkpoint_000337/lora.safetensors --scaling 2.0 \
    --out ckpt/duplex-brain-stage2.safetensors

--scaling 2.0 is the run's lora_scaling, applied directly, not divided by rank. Keep the generated .depq.json and .vocab.json sidecars beside the weights.

⚠️ Two failure modes that load cleanly and are silently wrong

The sidecars are not optional. Loading the dep_q=8 weights without .depq.json silently duplicates agent heads into partner heads; without .vocab.json, <instr_start> silently aliases BOS. Keep both files beside the .safetensors.
The injection mode is required and cannot be inferred. An adapter's tensors cannot distinguish ins_token from inline. Always pass mode=ins_token; a wrong mode produces plausible-sounding, wrong output rather than an error.

Ids ≥ 32000 are not SentencePiece pieces — id_to_piece(32000) raises. Render them from the .vocab.json sidecar, never through the tokenizer.

Caveats

Free-run speech quality was never systematically evaluated during training. The documented failure mode for this recipe family is the model going near-mute (13.2 → 3.9 words/clip) while eval loss improves. Step 337 has downstream benchmark numbers, which is stronger evidence than any offline metric here — but run audio_health (≥6 seeds; the failure is ~50% per-rollout stochastic) before trusting a checkpoint for generation.
No directive probe was run on any stage-2 checkpoint. Whether stage 2 preserved stage-1's steering is unmeasured, and stage 2 drops Fisher replay by design.
Both runs were cut by a 3-day wall clock. Stage 1 stopped at 7000 of a planned 10000 steps, so the OneCycle schedule annealed to ~4.5e-7 rather than to 0. Stage 2 used batch_size 2 (vs 4 in the reference recipe).
Stage-1-only models never saw a conv-synth goal card. Prompt them with a caller-seat task prompt (they are the user side, not an assistant) and expect loose adherence to goal lists.
Eval delivers all instructions deterministically (no dropout), so eval loss is self-consistent across checkpoints but not comparable to training loss.
Trained on Fisher (LDC) and Seamless Interaction; neither corpus is redistributed here.
Base weights are NVIDIA's PersonaPlex — their licence governs the base model.

Provenance

Code: MagicLuke/duplex-online-if. Training data derived from Fisher (LDC), Seamless Interaction and the conv-synth goal-arc corpus; instruction annotations from MagicLuke/{fisher,seamless}-turn-instructions-v1. Benchmark outputs: MagicLuke/ifbench-conversations-v1 and MagicLuke/fdb-v1-outputs-v1.

Downloads last month: -

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for MagicLuke/oif-instok-sft

Base model

kyutai/moshiko-pytorch-bf16

Finetuned

nvidia/personaplex-7b-v1

Adapter

(14)

this model

MagicLuke
/

oif-instok-sft

You need to agree to share your contact information to access this model

oif — online-instruction-following SFT checkpoints

Contents

`stage2_ins_stage1f/checkpoint_000337`

IF-Bench v1.3 — 200 tasks, judge `Qwen/Qwen3.8-27B`

FDB v1.0 — 727 stimuli

Training-data overlap with IF-Bench — measured

`stage1f_final_ins/checkpoint_004000`

Measured behaviour, including what is wrong with it

Usage

⚠️ Two failure modes that load cleanly and are silently wrong

Caveats

Provenance

Model tree for MagicLuke/oif-instok-sft

You need to agree to share your contact information to access this model

oif — online-instruction-following SFT checkpoints

Contents

stage2_ins_stage1f/checkpoint_000337

IF-Bench v1.3 — 200 tasks, judge Qwen/Qwen3.8-27B

FDB v1.0 — 727 stimuli

Training-data overlap with IF-Bench — measured

stage1f_final_ins/checkpoint_004000

Measured behaviour, including what is wrong with it

Usage

⚠️ Two failure modes that load cleanly and are silently wrong

Caveats

Provenance

Model tree for MagicLuke/oif-instok-sft

`stage2_ins_stage1f/checkpoint_000337`

IF-Bench v1.3 — 200 tasks, judge `Qwen/Qwen3.8-27B`

`stage1f_final_ins/checkpoint_004000`