inferenceProviderMapping is empty — no serverless
Inference Provider serves it. HF Spaces hardware tops out at ZeroGPU's 96 GB, so the weights
cannot be loaded here. Everything on this page instead runs at zero GPU cost:
the encoder is the model repo's own Python running in your browser via Pyodide, the charts are
the model card's published numbers, and the Playground talks to an endpoint you supply.
Model summary
| Architecture | Mixture-of-Experts (MoE), DeepseekV4ForCausalLM |
| Layers | 43 |
| Attention | MLA (low-rank q/o factors) with sparse token indexer, 64 heads, hidden 4096 |
| Experts | 256 routed + 1 shared, 6 selected per token, MoE intermediate 2048 per expert |
| Vocabulary | 129,280 |
| Context length | 1,048,576 tokens (1 M) |
| Quantization | block-FP8 attention (e4m3, 128×128) / packed-FP4 experts |
| Speculative decoding | DSpark multi-token module (inherited, fully functional) |
| Params / checkpoint | 304.18 B params · 166.9 GB · 48 shards · 1.48 B params kept in BF16, 6.30 B in FP8, 296.35 B expert params in packed I8/FP4 |
| Base model | deepseek-ai/DeepSeek-V4-Flash-0731 |
| Adaptation | 2× SFT + 1× KTO LoRA, trained in sequence, merged as full deltas — attention only |
| License | MIT |
What changed vs the base model
Headline deltas from the card: +14.3 LiveBench agentic coding, +10.1 Terminal Bench 2.1 (DeepSeek harness), +13.7 AutomationBench, +19.1 NL2Repo-Bench, +3.2 LiveBench overall — see the Benchmarks tab.
Training was 2 SFT steps plus a KTO step: three LoRA adapters trained in sequence and merged as full (untruncated) deltas into only the 129 MLA attention factor matrices. Experts, router, embeddings and the DSpark speculative module stay byte-identical to the official release, packed FP4 included — which is why any stack that serves the base release serves Smaug-Flash unmodified. Touched matrices were re-quantized to the checkpoint-native block-FP8 format.
Known behaviors (from the card)
- More decisive than its base: finishes tasks in fewer, denser turns and submits sooner.
- Under tight wall-clock budgets that wins several timeout-bound tasks the base loses.
- Under hidden-test grading it can submit at ~98 % spec coverage without spending the remaining budget on self-verification. If your use case rewards exhaustive verification, prompt for explicit self-testing before completion.
Gotchas worth knowing before you serve it
- No Jinja chat template ships with the repo. You must encode messages yourself —
that is what the Prompt encoder tab does, using the repo's own
encoding_dsv4.pyunmodified. reasoning_effort∈ {low,high,max};Noneis treated aslow. Forhigh/maxthe card recommends a maximum output length of 384 K tokens.- Sampling defaults:
temperature=1.0,top_p=0.95. - Loading needs
trust_remote_code. - Wall-clock task budgets on timeout-bounded benchmarks are effectively inference-compute budgets — benchmark on serving comparable to what you deploy.
Sources
Smaug-Flash vs its base
Higher is better. Δ is shown in each axis label. Rows marked * use the base model's
vendor-reported score for the identical public task set; unstarred rows are paired runs under the
same harness (card footnote).
| Benchmark | Smaug-Flash | DeepSeek-V4-Flash-0731 | Δ |
|---|
LiveBench 2026-06-25 category profile
Scores 0–100; overall = mean of the seven category averages. Same harness, same questions, paired against the base model.
| Category | Smaug-Flash | Base | Δ |
|---|
Original figures from the model card
No chat template — so here is the real one
abacusai/Smaug-Flash ships no Jinja chat template, so
tokenizer.apply_chat_template() is not an option. The repo's
encoding/encoding_dsv4.py
(MIT) is vendored into this Space byte-for-byte unmodified and executed below
inside Pyodide — real CPython, in your browser, no server round-trip.
reasoning_effort | low = no preamble (0 chars) · high = 476 chars · max = 526 chars of injected instruction. None ⇒ low. Only effective in thinking mode, only on the first message. |
thinking_mode | thinking opens <think> after <｜Assistant｜>; chat closes it immediately with </think> so the model answers directly. |
| Special tokens | <｜begin▁of▁sentence｜> · <｜User｜> · <｜Assistant｜> · <｜end▁of▁sentence｜> · <think>/</think> · <｜latest_reminder｜> · |DSML| for tool calls |
Delimiter nuance worth knowing: the repo's encoding_dsv4.py
and tokenizer.json delimit special tokens with U+FF5C
(｜, a fullwidth reverse solidus) rather than ASCII | — the two are
near-identical in most fonts, which is why hand-typed pipes silently fail to parse. Copy the tokens
from this page or from the repo, don't retype them; the decoder below repairs typed ASCII pipes
automatically and tells you when it did.
Python runtime not loaded yet — click Encode prompt and it will start (≈ 10 MB, one time).
Tool schemas (optional JSON)
Declaring tools switches the encoder's drop_thinking off by itself —
the model was trained to keep its reasoning across tool loops.
Decode a completion back into OpenAI format
The parser only accepts well-formed output — the card is explicit that it does not
recover from malformed generations: the think block must be closed with </think>
and the turn must end with <｜end▁of▁sentence｜>.
Chat with Smaug-Flash — through an endpoint you own
This page is static: there is no backend, no GPU and no proxy. Your browser calls an OpenAI-compatible endpoint directly. Serve the model with vLLM or SGLang (see Deploy & size) and paste the base URL below. Anything OpenAI-compatible works — including a model on the HF router, for a plumbing check.
Access-Control-Allow-Origin; vLLM allows all origins by default, most hosted APIs do too,
but a bare python -m http.server does not. (2) HTTPS: this page is served over
HTTPS, so a plain http:// endpoint will be blocked as mixed content — terminate TLS
(tunnel, reverse proxy) or open this page over http://localhost from the repo.
Your key lives only in this tab: it is sent to the endpoint you type and nowhere else.
Sampling & reasoning
Nothing here yet — set an endpoint, then send a message.
Serve the real thing
Any stack that runs deepseek-ai/DeepSeek-V4-Flash-0731 runs Smaug-Flash unmodified —
only the attention factors changed, so the FP4 expert kernels and the DSpark speculative path are
untouched.
SGLang ≥ 0.5.15 (Blackwell) — the card's primary recipe
vLLM (single 4×GB300 node)
Reference transformer path (the repo's own inference/ folder)
Python client (no chat template, so encode first)
Serving settings the card is explicit about: reasoning_effort=max,
temperature=1.0, top_p=0.95, fp8 KV cache, full 1 M context, and up to
384 K output tokens for high/max effort. Local weight conversion and interactive
demos live in the repo's inference/ folder.
Memory calculator: where would this even run?
Parameter split is computed from config.json: 43 layers × (256 routed + 1
shared) experts × 3 matrices × 4096 × 2048 = 278.1 B expert parameters; the
remaining 26.1 B are attention, embeddings and the DSpark module. Total
304.18 B, matching the Hub's safetensors metadata.
Does it fit anywhere on Spaces?
| Flavor | Hardware | VRAM | Price | vs budget |
|---|
Notes that change the answer
- The official recipes use
tp=4/data-parallel-size 4with expert parallelism and theflashinfer_mxfp4/deep_gemm_mega_moeMoE backends — the packed-FP4 experts need kernels that exist only on recent-GPU stacks (SGLang ≥ 0.5.15 on Blackwell). - “Just quantize harder” is not a free win: the card warns that re-quantizing merged weights degrades constrained instruction-following.
- KV-cache size is deliberately excluded — MLA's compressed ratios in
config.jsonmake a naive per-token number misleading; use the headroom slider instead. - Paid Spaces tiers bill per hour while the Space runs. ZeroGPU (this Space's flavor) is still capped at 96 GB — far below the 154 GB native budget above.
Cheaper routes that work today
- Tunnel a deployment (cloudflared / ngrok / tailscale funnel) and paste the public HTTPS URL into the Playground.
- Open this page locally (
python -m http.serverin the repo) and talk tohttp://127.0.0.1:8000/v1with no TLS or CORS friction. - Community GPU grant if you want the Space itself to serve it: open a
discussion titled
Apply for a GPU community grant: Personal projecton the Space, explaining it is a public open-source demo of an MIT-licensed 304 B model that cannot fit in ZeroGPU's 96 GB. Approval typically takes days; hardware is then attached automatically.