State-of-the-art audio decisions in 4B parameters, and it sees and reads too.
60.1 Earshot audio skill, ahead of Gemini 4 Argon (59.4) and Gemini 3.8 Flash (50.5), and better calibrated than both.

1. Model Introduction

Smaug-Decisions is a 4B decision model that hears, sees and reads. A fine-tune of Google's Gemma 4 E4B-it by Abacus.AI, it answers a typed question about audio, an image or text with a calibrated probability per option, in one forward pass and with no generated text.

Four example decisions over audio, an image and text, each answered with a probability per option in one forward pass

60.1 Earshot audio skill, three input types, one forward pass per decision, 0.069 calibration error

It is the only 4B decision model with results on all three input types. On audio it is ahead of the frontier APIs: 60.1 skill on Earshot, our 6,671-item audio decision benchmark, against 59.4 for Gemini 4 Argon and 50.5 for Gemini 3.8 Flash, and better calibrated than both (ECE 0.069 vs 0.123 and 0.204). On the public Jev Decision Index it scores 57.9 on images and 56.6 on text, the best Gemma 4 E4B on both.

import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("abacusai/Smaug-Decisions", allow_patterns=["smaug_decide.py"]))
from smaug_decide import SmaugDecide                                   # ships with this repo

model = SmaugDecide.from_pretrained("abacusai/Smaug-Decisions")
model("call.wav", "Has the speaker finished their turn?")               # -> {'Yes': 0.93, 'No': 0.07}

Every 4B decision model by input type: Smaug-Decisions is the only one with audio, image and text results

2. Highlights

Ahead of the frontier APIs on audio, and better calibrated, so the probabilities work as thresholds.
Hears, sees and reads: audio up to 30 s, images and text through one prompt and one readout.
One pass, no decoding: about 200 ms per audio decision on an H100.
Runs on phones: Smaug-Decisions-LiteRT is 3.45 GB, keeps 59.4 audio skill and answers a text question in about 0.6 s on an iPhone 14 Pro Max, fully offline.

3. Model Summary

Architecture	Gemma 4 E4B-it (language model + audio and vision encoders); letter-logit readout
Parameters	4B effective, 8.6B stored (3.5B of the stored weights are embedding tables that are looked up, not computed; the Hugging Face counter shows the stored total)
Input	Audio up to 30 s (16 kHz mono), an image (560 image tokens), or text, plus a question with 2 to 20 options
Output	A calibrated probability per option (softmax for one answer, sigmoid for select-all)
Hardware	NVIDIA GPU (bfloat16), Apple Silicon (MPS) and CPU through Transformers; phones through Smaug-Decisions-LiteRT
Precision	bfloat16
Formats	Transformers (repo root); LiteRT-LM for phones (Smaug-Decisions-LiteRT)
Base Model	`google/gemma-4-E4B-it`
Adaptation	Full fine-tune

4. Evaluation

Audio: Earshot

Earshot is our audio decision benchmark: 6,671 items and 17.2 hours of audio in four tracks: conversation dynamics (turn completion, next speaker, speaker count, speech activity; from VoxConverse, AMI and Smart Turn), external turn completion (Smart Turn v3.2 test), sound and music (genre, instrument, tempo, scene, sound events) and a small pilot track. Audio skill is the mean task skill per track, averaged over the tracks: 0 is guessing, 100 is perfect. No benchmark recording or item is in the training data.

Earshot audio skill per track, and calibration error

Earshot table

	Smaug-Decisions	Gemini 4 Argon (API)	Gemini 3.8 Flash (API)	AudioJev (Qwen2.5-Omni-3B)	Gemma 4 E4B-it (zero-shot)
Earshot (audio decisions)
Audio skill (headline) ↑	60.1	59.4	50.5	43.6	5.0
ECE ↓	0.069	0.123	0.204	0.191	0.464
Per track (skill) ↑
Conversation dynamics (3,000)	48.2	49.6	48.5	26.7	3.6
External turn completion (1,000)	80.8	76.5	81.8	62.0	11.2
Sound & music (1,115)	66.1	64.7	42.4	57.6	5.2
Pilot (80)	45.2	47.0	29.1	28.2	0.1

Bold = best in the row. Shading is relative within each row (darker = better). Every model scored on the same 6,671 items with the same harness; the APIs received the audio and returned JSON probabilities. Skill = (accuracy − chance) / (1 − chance), times 100.

Need it faster? Smaug-Audio-Decisions-0.7B scores 53.0 on Earshot at 9.7 ms; in front of this model as a cascade it matches this model's accuracy (60.1) at 130 ms instead of 210 ms.

Images and text: the public Jev Decision Index

The matrix at the top has the headline numbers: 57.9 on images and 56.6 on text, the best Gemma 4 E4B on both (Winnow-E4B, the other Gemma 4 E4B fine-tune on the boards: 48.1 and 41.7). Images: our run of the official vision kit (0.4, public sets) at 560 image tokens.

Full Decision Index breakdown

Text decision skill per area

	Smaug-Decisions (audio + image + text)	Perplexity Decider Flash (4B)	vLLM-SR d3-mini (4B)	RSI-Jev v6.1 VL (4B)	Winnow (Gemma 4 E4B)
Jev Decision Index 0.4.1, text (same 35 public benchmarks)
Public skill ↑	56.6	61.8	56.7	52.9	41.7
Knowledge & reasoning	35.1	46.0	44.3	35.2	25.3
Language	66.4	67.2	60.4	59.2	47.5
Retrieval	64.7	65.2	63.6	57.1	43.8
Tools	72.9	81.0	72.0	68.4	62.5
Arts	41.6	46.6	37.7	45.7	26.9
Earshot (audio decisions)
Audio skill ↑	60.1	—	—	—	—

Bold = best in the row. Other models: the public Decision Index board as of 2026-10-28 (text 0.4.1). Smaug-Decisions: our runs of the official Decision Index kit (text on a 3,000-row sample of the public suite, compared on the same 35 benchmarks); self-reported, not board submissions. — = no published result.

5. Training Approach

Readout, not generation. The answer is read from the LM-head logits of the option letters at the answer position (tanh softcap 30, then softmax; select-all uses a sigmoid with one learned threshold). A decision costs one prefill.
Decision-tuned and calibrated. Full fine-tune of Gemma 4 E4B-it on a broad mix of typed decision tasks over audio, images and text, with calibration as an explicit objective and options shuffled so the model learns the options, not their positions.

6. Limitations

Decisions, not text: each call answers one typed question with a probability per option; the model does not write text.
Per request: one audio clip of up to 30 s (longer audio keeps its last 30 s), one image at 560 image tokens, up to 20 options.
Thresholds: set decision thresholds on your own data for high-stakes uses.

7. Usage

pip install "transformers>=5.20" torch soundfile pillow huggingface_hub

import sys
from huggingface_hub import snapshot_download
sys.path.insert(0, snapshot_download("abacusai/Smaug-Decisions", allow_patterns=["smaug_decide.py"]))
from smaug_decide import SmaugDecide

model = SmaugDecide.from_pretrained("abacusai/Smaug-Decisions")      # CUDA, Apple Silicon (MPS) or CPU, picked automatically
model("call.wav", "Has the speaker finished their turn?")                                  # predicate -> {'Yes': p, 'No': 1-p}
model("song.wav", "Which genre is playing?", ["rock", "jazz", "classical"])                 # choice -> probabilities sum to 1
model("room.wav", "Which sounds are present?", ["speech", "music", "alarm"], multi=True)    # select-all -> one probability each
model("receipt.jpg", "Does the total exceed $50?")                                          # images
model("The order never arrived.", "How upset is the customer?", ["calm", "annoyed", "angry"],
      descriptions=["neutral tone", "complains", "threatens to leave"], rules="Judge the tone, not the facts.")

from_pretrained downloads the checkpoint (about 16 GB). Any runtime can reproduce the readout: run the prompt in smaug_decide.render, take the LM-head logits of the tokens A..T at the last position, apply the softcap from readout.json, then softmax (one answer) or sigmoid(z - b) with b from readout.json (select-all). With plain Transformers, AutoModelForMultimodalLM.from_pretrained("abacusai/Smaug-Decisions", dtype="bfloat16") loads the weights.

On phones

Smaug-Decisions-LiteRT is this model in one 3.45 GB file for Google's on-device runtime LiteRT-LM, running on the phone GPU. At about a fifth of the size it keeps almost all of this model's quality. It runs in the Cascade app on iPhone and Android next to Smaug-Audio-Decisions-0.7B, which answers the easy audio questions first. App source: github.com/abacusai/Smaug-Decisions-Mobile.

	Smaug-Decisions (16 GB)	Smaug-Decisions-LiteRT (3.45 GB)	change
Earshot audio skill	60.1	59.4	−0.7
Earshot calibration error (ECE, lower is better)	0.069	0.074	+0.005
Decision Index text	56.6	53.9	−2.7
Decision Index images, 560 image tokens^*	57.2	55.1	−2.1
Decision Index images, 280 image tokens^*	54.8	51.8	−3.0

Same items, same harness; the phone-build scores come from a PyTorch replica of the file's quantized weights. ^*10 of the 11 public vision sets (Winoground left out; this model's 11-set index is 57.9).

iPhone 14 Pro Max (6 GB RAM, phone GPU)	measured
first load (one time) / later loads	13-21 s / 0.48 s
text decision	~0.6 s
audio decision	~1.6 s
image decision (280 image tokens)	~7.4 s
peak app memory, with Smaug-Audio-Decisions-0.7B loaded	1.35 GB
storage	the 3.45 GB file + a ~2.4 GB cache written on first load

8. License

Apache 2.0, the same license as the base model, Gemma 4 E4B-it.

9. Citation

@misc{abacusai2026smaugdecisions,
  title  = {Smaug-Decisions: calibrated typed decisions over audio, images and text},
  author = {Abacus.AI},
  year   = {2026},
  url    = {https://huggingface.co/abacusai/Smaug-Decisions}
}