Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
BitKnight 
posted an update about 5 hours ago
Post
45
DuplexJev: a System One model for speech 🎧

Jev made "System One" decision models the hot idea of the season: state in, typed probabilistic decisions out, many questions in one pass. Text has Kev, Openjev and Jeva; video has VideoJev. Voice agents need it most, yet a text decision model only sees a transcript, and the transcript is where tone disappears.

DuplexJev takes raw audio as its state. A frozen ASR encoder feeds a frozen LLM through a small connector; every question is a choice question read as one token, nothing decoded.

⚡ Fast: 10 decisions in 92 ms on one H200 (transcribe + LLM: 1,978 ms), ~170 decisions/s per GPU, under $0.50 per million decisions (4B). A new decision is one line of text.

🏆 Hears what the transcript drops: state of the art in paralinguistic understanding. LISTEN (EACL 2026) + StressTest + MSPB, 1,128 identical questions:
DuplexJev-32B-Para v1.3: 50.9
Gemini 3.8 Flash: 46.9
Qwen3-Omni-30B: 44.6
GPT-audio-1.5: 40.1
Gemini 3.8 Live: 38.9
GPT-realtime-2.1: 34.2
Full LISTEN set: 38.9 vs 33.0 best published.

How it learned to listen: speech LLMs trained by transcript distillation imitate a teacher that never heard a voice (gender ~55%, emotion ~28%). Training the answer token on the correct answer instead takes both to ~90%.

Paper: Batched Speech Decisions Without Decoding: Single-Token Supervision Lets a Frozen LLM Hear Beyond the Transcript (2610.02638)
Models: adventists-ai/duplexjev-6acc69e95e4fc69acd09adaf
Demo: BitKnight/DuplexJev
Code + every model's raw answers: https://github.com/adventists-ai/duplexjev

(Independent project by Adventists AI, Zhejiang University and University of Virginia; not affiliated with TypeSafe.)

Have you tested whether the decision and confidence stay stable when you shuffle the answer options? I would reuse the same audio clips and map the option letters back to their labels, then report label flips and confidence shifts. That would help check how much the readout depends on option position.

·

Good idea, so we just ran it: the answers don't depend on option position. Same clips, options shuffled, letters mapped back to labels (DuplexJev-32B-Para v1.3, one order per request, no averaging):

  • 29 clips; 4-way emotion and 4-way turn state with 15 distinct option orders each, gender with both orders.
  • 98.5% of answers are identical across all orders (emotion 686/696, turn 685/696, gender 58/58).
  • When the model is confident (≥ 0.8), zero flips, and the winning label's probability barely moves (median change 0.01 for emotion, 0.04 for turn).
  • The few flips all come from 2 genuinely ambiguous clips per question, where the model was only 0.55–0.67 sure to begin with.

For benchmarks we still average over a few orders, which settles even those borderline clips (29/29 match) at almost no cost, since the orders share the audio prefix. We'll run the same check on the full benchmark set and add it to the repo. Thanks for the suggestion!