Sophon-Realtime-4B-Flash-MLX

No GPU? Try it through the hosted API: sign up and get 3 hours of free voice chat (OpenAI Realtime-compatible, docs).

Version 1.5.0 (2026-11-18) · changelog · code: mac/ · CUDA build: Sophon-Realtime-4B-Flash · 32B Mac build: Sophon-Realtime-32B-Flash-4bit-MLX · 中文:README_CN.md

The full-duplex speech model Sophon-Realtime-4B-Flash, running locally on an Apple silicon Mac with 16 GB or more (MLX). You talk, it decides by itself when you have finished, answers with its own voice and sound tags, stops when you interrupt, and carries on when you just say "mm-hmm". Hearing, turn-taking and speaking are all done by the model on the Mac's GPU: no cloud, and no speech-to-text step in between.

The thinker's language model is quantized to 8 bits by mlx_lm (group 64); the audio tower, projector and voice stage are the same weights as the CUDA build.

4B or 32B on a Mac

4B (this repo) 32B
Unified memory 16 GB works (about 11 GB in use while talking); 24 GB or more recommended 32 GB works, 36 GB or more recommended (about 26 GB in use)
Download about 9 GB about 24 GB
You stop talking → it starts speaking (M4 Max) 0.3–0.5 s when it is sure you have finished 0.9–1.1 s
Knowledge and reasoning much weaker (VoiceBench 62.4 on the CUDA build) strong (82.8)

The 4B brain keeps the conversation skills (turn-taking, back-channels, barge-in, sound tags) but knows and reasons much less, and can state wrong facts with confidence. It is for local demos, companions and characters; use the 32B build or the hosted API when answers must be right.

Requirements

  • Apple silicon (M1 or later), macOS 14 or later, 16 GB unified memory or more (with 16 GB, close other apps)
  • About 9 GB download
  • A microphone. Mac Studio and Mac mini have none built in: plug in a USB microphone or a headset. Speakers are fine, echo cancellation is built in

Quick start

git clone https://github.com/adventists-ai/sophon-realtime && cd sophon-realtime
SIZE=4B bash mac/install.sh       # Python environment + this repo's weights, into ~/sophon-mac/models-4b
~/sophon-mac/sophon-4b            # talk in Chinese
~/sophon-mac/sophon-4b --lang en  # talk in English

Options: --no-aec (headset, no echo cancellation), --in / --out (audio devices, --list shows them), --voice. The 32B and 4B builds can be installed side by side.

Turn-taking

The 4B brain answers the server's four-option turn question ("finished / still talking / back-channel / hesitating"), asked in its own short prompt; the 32B Mac build's shorter two-option question is tuned for 32B and does not suit 4B (turn_question in config.json selects it). Accuracy on a sample of the Smart Turn v3.2 test set (300 clips per language, half of them finished; last 8 s of each):

English acc. / AUC Chinese acc. / AUC
4B, this repo 90.0 / 0.961 89.7 / 0.954
4B with the 32B build's short question 72.3 / 0.888 74.0 / 0.851
32B Mac build (same sample) 91.7 / 0.961 92.7 / 0.973

When the model is not sure you have finished, it waits up to 2 s of silence before answering.

Status and limits

  • Tested on an M4 Max (36 GB) with real-time simulated audio input, and with MLX memory capped at 10.5 GB (what a 16 GB Mac gives the GPU): the whole conversation ran, peak memory 11.3 GB. Not yet tried on a physical 16 GB Mac; reports are very welcome in issue #1.
  • One conversation at a time. Camera input, tool calls and zero-shot voice cloning are not in the Mac version yet (built-in voices only).

Files

Folder What
thinker_mlx/ the thinker's language model for mlx_lm, 8-bit
ear/ the thinker's audio tower and projector
talker/ the voice stage (same as Flash)
voices/ built-in voices

License: code Apache-2.0; model weights Apache-2.0, see NOTICE for sources.

Downloads last month
116
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for adventists-ai/Sophon-Realtime-4B-Flash-MLX

Finetuned
(2)
this model

Collection including adventists-ai/Sophon-Realtime-4B-Flash-MLX