Papers
arxiv:2611.18330

Marimba: A System One Model for Speech

Published on Nov 13
· Submitted by
Solano Amadori Todeschini
on Nov 17
Authors:

Abstract

Marimba uses frozen Qwen3-ASR features and lightweight readouts to make lexical, turn-end, language, speaker-change, and personal-information decisions from speech, achieving strong detection and calibration on conversational and read-speech benchmarks.

Speech applications require decisions about term presence, turn completion, and language within an audio window. We present Marimba, an interface for typed speech decisions built on frozen Qwen3-ASR 0.6B features. Lexical questions use zero-shot scoring of ASR transcripts; non-lexical questions use a question-conditioned head and a separate layer-18 turn-end readout. Language identification uses the ASR model's own language scores. On private conversational test sets, decoder AUROC is 0.80-0.92; on public read speech it is 0.92-0.97. Speech Commands four-way accuracy is 0.93; binary lexical ECE is at most 0.08. Personal-information detection, which passes the greedy transcript to a private, curated PII pipeline, reaches item-level F1 0.904 on synthetic Brazilian Portuguese speech (1.000 on reference transcripts). Layer-12 speaker readouts detect speaker change with AUROC 0.966-0.980 on AMI and VoxConverse, above an ECAPA baseline, and count speakers (0, 1, 2, or 3+) more accurately than pyannote 3.1. The turn-end readout reaches AUROC 0.930 versus Smart Turn v3.2 at 0.982 on Smart Turn's own test set. On a subset of ICSI meetings labeled by next speaker, AUROC is 0.716 versus 0.712; the paired confidence interval includes zero.

Community

Paper author Paper submitter

Marimba is a System One model for speech: ask typed questions about live audio, get typed answers back every 250 ms.

Is someone speaking? Did the turn end? Which language? Did they say "refund"? Did the speaker change? How many speakers?
Answers come back as yes/no, a choice, or a score.

  • One frozen Qwen3-ASR-0.6B serves every question, so it runs on a laptop CPU.
  • Counts speakers better than pyannote 3.1 on AMI and VoxConverse; 100% language ID on 300 FLEURS clips.
  • Open weights, benchmark and code (Apache-2.0).

Latency (p50 per question, 8 s window, batch 1):

Question Mac M4 Pro CPU NVIDIA L4 GPU
Turn end ~100 ms ~12 ms
Voice activity ~105 ms ~14 ms
Language ~1.3 s ~150 ms
Term ("did they say X?") not measured ~1.4 s

Turn end and voice activity are fast on CPU; term detection needs ASR decoding, so a GPU helps there. Protocol: https://github.com/Charcot-AI/marimba-api/blob/main/docs/benchmarks.md

Try it live: https://huggingface.co/spaces/charcot-ai/marimba
Model: https://huggingface.co/charcot-ai/marimba
Benchmark: https://huggingface.co/datasets/charcot-ai/marimba-bench
Code: https://github.com/Charcot-AI/marimba-api

Author here, ask me anything!

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2611.18330
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Datasets citing this paper 1

Spaces citing this paper 1

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.