abacusai/Smaug-Audio-Decisions-0.7B

---

Audio decisions that beat a frontier API, in 10 ms.
53.0 Earshot audio skill, ahead of Gemini 3.8 Flash (50.5), from a 744M model small enough to ship inside a phone app.

1. Model Introduction

Smaug-Audio-Decisions-0.7B is the 10 ms audio decision tier by Abacus.AI: a fine-tune of Google's EmbeddingGemma 2 (744M parameters, audio tower included) that answers a typed question about a clip with a calibrated probability per option, in one forward pass, on a GPU, a Mac or fully offline on an iPhone.

It scores 53.0 on Earshot, our 6,671-item audio decision benchmark, ahead of Gemini 3.8 Flash (50.5) and the 3B AudioJev (43.6). Put in front of Smaug-Decisions as a confidence-gated cascade, it matches the big model's accuracy (60.1) at 130 ms instead of 210 ms.

import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("abacusai/Smaug-Audio-Decisions-0.7B", allow_patterns=["smaug_decide.py"]))
from smaug_decide import SmaugDecide                                   # ships with this repo

model = SmaugDecide.from_pretrained("abacusai/Smaug-Audio-Decisions-0.7B")
model("call.wav", "Has the speaker finished their turn?")               # -> {'Yes': 0.91, 'No': 0.09}

2. Highlights

• Calibrated: ECE 0.086, against 0.204 for Gemini 3.8 Flash.
• Fast everywhere: 9.7 ms per decision on an H100, about 0.3 s on an iPhone 14 Pro Max, fully offline.
• Big-model accuracy at 62% of the latency as a cascade in front of Smaug-Decisions.
• Ready to load: Transformers weights at the root, int8 Core ML models in coreml/ that score the same as float (53.0).

3. Model Summary

Architecture	EmbeddingGemma 2 (text trunk + audio and vision towers); linear head at a marker token per option
Total Parameters	744M
Input	Audio up to 30 s (16 kHz mono), plus a question with 2 or more options (images and text are accepted, see below)
Output	A calibrated probability per option (softmax for one answer, sigmoid for select-all)
Hardware	NVIDIA GPU, Apple Silicon, CPU; iPhone via Core ML
Precision	float32 weights (loaded in bfloat16 on GPUs); int8 Core ML in coreml/
Formats	Transformers (root), Core ML (coreml/)
Base Model	google/embeddinggemma-2
Adaptation	Fine-tune with a linear marker head

4. Evaluation

Audio: Earshot

Earshot is our audio decision benchmark: 6,671 items, 17.2 hours of audio, four tracks. Skill: 0 is guessing, 100 is perfect; ECE is the calibration error of the top probability. Every model went through the same harness, the APIs with audio in and JSON probabilities out.

All models tested

model	audio skill	ECE	conversation dynamics	ext. turn completion	sound & music	pilot
Smaug-Decisions (4B)	60.1	0.069	48.2	80.8	66.1	45.2
Gemini 4 Argon (API; 5 of 6,671 refused on safety)	59.4	0.123	49.6	76.5	64.7	47.0
Smaug-Audio-Decisions-0.7B (744M)	53.0	0.086	42.4	65.8	58.0	45.8
Gemini 3.8 Flash (API; 36 refused on safety)	50.5	0.204	48.5	81.8	42.4	29.1
AudioJev (Qwen2.5-Omni-3B)	43.6	0.191	26.7	62.0	57.6	28.2
Gemma 4 E4B-it zero-shot (same prompt)	5.0	0.464	3.6	11.2	5.2	0.1
cascade: this tier, escalate to Smaug-Decisions below 0.9	60.1	0.071	48.2	80.8	66.1	45.2

The cascade

Mean latency per decision on an H100, including escalations.

confidence threshold	escalated to Smaug-Decisions	audio skill	ECE	mean latency
fast tier alone	0%	53.0	0.086	9.7 ms
0.8	46%	59.2	0.071	101 ms
0.85	52%	59.4	0.072	114 ms
0.9	60%	60.1	0.071	130 ms
0.95	72%	60.2	0.069	153 ms
Smaug-Decisions alone	100%	60.1	0.069	210 ms

Text and images

The model is built for audio: text decisions are close to chance (6.9 skill on the public Jev Decision Index text sample) and image decisions are not benchmarked. It knows this (at 0.9 it escalates 98% of text questions), so a cascade answers text at Smaug-Decisions' level.

On phones: int8

The phone apps run this model with int8 weights and an int8 embedding table (Core ML from coreml/ on iOS, int8 ONNX on Android). It loses nothing measurable: on Earshot the Core ML int8 weights score 53.0 skill with ECE 0.090, against 53.0 and 0.086 in float, and the two pick the same answer on 99.6% of items (PyTorch replica of the int8 weights, same harness).

Memory and speed

On an iPhone 14 Pro Max (6 GB) the Cascade app answers in 0.27-0.37 s per decision, fully offline. App size with this model inside: iOS 602 MB, Android about 582 MB (Play download, arm64).

5. Training Approach

• Marker readout. Each option is preceded by a marker token (<unused0>); a linear head scores the hidden state at each marker (softmax for one answer, sigmoid per option for select-all). One prefill, no decoding.
• Decision-tuned and calibrated. Fine-tuned from EmbeddingGemma 2 on typed decision tasks, to sit in front of Smaug-Decisions: it answers the confident cases and hands the rest up.

6. Limitations

• Audio first: text and image questions go to Smaug-Decisions; the cascade routes them automatically.
• Per request: one clip of up to 30 s (the phone app records 10 s).
• Thresholds: set the escalation threshold on your own data.

7. Usage

pip install "transformers>=5.20" torch soundfile pillow huggingface_hub

import importlib.util
from huggingface_hub import snapshot_download

def load(repo):                                                   # each tier ships its own smaug_decide.py
    path = snapshot_download(repo, allow_patterns=["smaug_decide.py"]) + "/smaug_decide.py"
    spec = importlib.util.spec_from_file_location(repo.split("/")[1].replace("-", "_"), path)
    mod = importlib.util.module_from_spec(spec); spec.loader.exec_module(mod)
    return mod.SmaugDecide.from_pretrained(repo)                  # CUDA, Apple Silicon (MPS) or CPU, picked automatically

fast = load("abacusai/Smaug-Audio-Decisions-0.7B")
p = fast("call.wav", "Has the speaker finished their turn?")                                # -> {'Yes': 0.91, 'No': 0.09}
fast("song.wav", "Which genre is playing?", ["rock", "jazz", "classical"])                   # choice -> probabilities sum to 1
fast("room.wav", "Which sounds are present?", ["speech", "music", "alarm"], multi=True)      # select-all -> one probability each

if max(p.values()) < 0.9:                                                                   # the cascade: 60.1 at 130 ms on an H100
    big = load("abacusai/Smaug-Decisions")
    p = big("call.wav", "Has the speaker finished their turn?")

from_pretrained downloads about 3 GB. Any runtime can reproduce the readout: render the prompt with smaug_decide.render, then logit_k = head.weight @ h[marker_k] + head.bias at each marker (head.pt), softmax or sigmoid.

iPhone / Core ML (coreml/, all int8 unless noted):

file	what it is
trunk_L512_int8, trunk_L1024_int8	the trunk for prompts up to 512 or 1,024 tokens
audio_10s_int8, audio_30s_int8	the audio encoder for 10 s or 30 s clips; audio_conv_10s_fp16 + audio_stack_10s_int8 is the 10 s encoder split for the Neural Engine
embed_tokens_fp16.bin	the token-embedding table
embed_tokens_int8.bin + _scales.bin	the same table at half the size (int8, one float32 scale per row), as the apps ship it
head.json, readout.json	the marker head and readout settings

The iOS app (recording, log-mel features, prompt and readout) is at github.com/abacusai/Smaug-Decisions-Mobile.

8. License

Apache 2.0, the same license as the base model, EmbeddingGemma 2.

9. Citation

@misc{abacusai2026smaugaudio,
  title  = {Smaug-Audio-Decisions-0.7B: a 10 ms calibrated decision tier},
  author = {Abacus.AI},
  year   = {2026},
  url    = {https://huggingface.co/abacusai/Smaug-Audio-Decisions-0.7B}
}

10. Contact Us

If you have any questions, please reach out at Abacus.AI.