Smaug-Flash-Explorer / index.html
xdLocutus's picture
Upload folder using huggingface_hub
37872b1 verified
Raw History Blame Contribute Delete
25.9 kB
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Smaug-Flash Explorer</title>
<meta name="description" content="Specs, benchmarks, the real prompt encoder and a bring-your-own-endpoint chat client for abacusai/Smaug-Flash." />
<link rel="stylesheet" href="styles.css" />
<script src="https://cdn.plot.ly/plotly-2.35.2.min.js" charset="utf-8"></script>
</head>
<body>
<header class="hero">
<h1>🐉 Smaug-Flash Explorer</h1>
<p class="sub">
Interactive companion for
<a href="https://huggingface.co/abacusai/Smaug-Flash" target="_blank" rel="noopener">abacusai/Smaug-Flash</a>
— Abacus.AI's agentic-coding fine-tune of
<a href="https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731" target="_blank" rel="noopener">deepseek-ai/DeepSeek-V4-Flash-0731</a>.
</p>
<div class="badges">
<span class="badge">304.18 B params</span>
<span class="badge">166.9 GB checkpoint · 48 shards</span>
<span class="badge">1,048,576 ctx</span>
<span class="badge">MIT</span>
<span class="badge">no GPU needed</span>
</div>
<p class="hublive" id="hub-live">loading live Hub stats…</p>
</header>
<nav class="tabs" role="tablist">
<button class="tab active" data-tab="overview" role="tab">🧭 Overview</button>
<button class="tab" data-tab="benchmarks" role="tab">📊 Benchmarks</button>
<button class="tab" data-tab="encoder" role="tab">🔤 Prompt encoder</button>
<button class="tab" data-tab="playground" role="tab">💬 Playground</button>
<button class="tab" data-tab="deploy" role="tab">🚀 Deploy &amp; size</button>
</nav>
<main>
<!-- ======================================================= Overview -->
<section class="panel active" id="tab-overview">
<div class="callout warn">
<strong>Why there is no in-Space inference.</strong> The checkpoint is
<strong>304 B parameters / 166.9 GB on disk</strong> (packed-FP4 experts + block-FP8 attention)
and its Hub <code>inferenceProviderMapping</code> is <strong>empty</strong> — no serverless
Inference Provider serves it. HF Spaces hardware tops out at ZeroGPU's 96 GB, so the weights
cannot be loaded here. Everything on this page instead runs at <strong>zero GPU cost</strong>:
the encoder is the model repo's own Python running in your browser via Pyodide, the charts are
the model card's published numbers, and the Playground talks to an endpoint you supply.
</div>
<h2>Model summary</h2>
<table class="kv">
<tr><td>Architecture</td><td>Mixture-of-Experts (MoE), <code>DeepseekV4ForCausalLM</code></td></tr>
<tr><td>Layers</td><td>43</td></tr>
<tr><td>Attention</td><td>MLA (low-rank q/o factors) with sparse token indexer, 64 heads, hidden 4096</td></tr>
<tr><td>Experts</td><td>256 routed + 1 shared, 6 selected per token, MoE intermediate 2048 per expert</td></tr>
<tr><td>Vocabulary</td><td>129,280</td></tr>
<tr><td>Context length</td><td>1,048,576 tokens (1 M)</td></tr>
<tr><td>Quantization</td><td>block-FP8 attention (e4m3, 128×128) / packed-FP4 experts</td></tr>
<tr><td>Speculative decoding</td><td>DSpark multi-token module (inherited, fully functional)</td></tr>
<tr><td>Params / checkpoint</td><td><strong>304.18 B</strong> params · <strong>166.9 GB</strong> · 48 shards · 1.48 B params kept in BF16, 6.30 B in FP8, 296.35 B expert params in packed I8/FP4</td></tr>
<tr><td>Base model</td><td><a href="https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731" target="_blank" rel="noopener">deepseek-ai/DeepSeek-V4-Flash-0731</a></td></tr>
<tr><td>Adaptation</td><td>2× SFT + 1× KTO LoRA, trained in sequence, merged as full deltas — attention only</td></tr>
<tr><td>License</td><td>MIT</td></tr>
</table>
<h2>What changed vs the base model</h2>
<p>
Headline deltas from the card: <strong>+14.3</strong> LiveBench agentic coding,
<strong>+10.1</strong> Terminal Bench 2.1 (DeepSeek harness), <strong>+13.7</strong>
AutomationBench, <strong>+19.1</strong> NL2Repo-Bench, <strong>+3.2</strong> LiveBench overall —
see the <a href="#" data-goto="benchmarks">Benchmarks</a> tab.
</p>
<p>
Training was 2 SFT steps plus a KTO step: three LoRA adapters trained in sequence and merged as
full (untruncated) deltas into <strong>only the 129 MLA attention factor matrices</strong>.
Experts, router, embeddings and the DSpark speculative module stay byte-identical to the official
release, packed FP4 included — which is why <em>any stack that serves the base release serves
Smaug-Flash unmodified</em>. Touched matrices were re-quantized to the checkpoint-native block-FP8
format.
</p>
<div class="callout">
<strong>Card caution:</strong> serve it in the checkpoint-native format. Re-quantizing the merged
weights into a different scheme at load time measurably degrades constrained instruction-following.
</div>
<h2>Known behaviors (from the card)</h2>
<ul>
<li>More decisive than its base: finishes tasks in fewer, denser turns and submits sooner.</li>
<li>Under tight wall-clock budgets that wins several timeout-bound tasks the base loses.</li>
<li>Under hidden-test grading it can submit at ~98 % spec coverage without spending the remaining
budget on self-verification. If your use case rewards exhaustive verification, <strong>prompt for
explicit self-testing before completion</strong>.</li>
</ul>
<h2>Gotchas worth knowing before you serve it</h2>
<ul>
<li><strong>No Jinja chat template ships with the repo.</strong> You must encode messages yourself —
that is what the <a href="#" data-goto="encoder">Prompt encoder</a> tab does, using the repo's own
<code>encoding_dsv4.py</code> unmodified.</li>
<li><code>reasoning_effort</code> ∈ {<code>low</code>, <code>high</code>, <code>max</code>};
<strong><code>None</code> is treated as <code>low</code></strong>. For <code>high</code>/<code>max</code>
the card recommends a maximum output length of <strong>384 K</strong> tokens.</li>
<li>Sampling defaults: <code>temperature=1.0</code>, <code>top_p=0.95</code>.</li>
<li>Loading needs <code>trust_remote_code</code>.</li>
<li>Wall-clock task budgets on timeout-bounded benchmarks are effectively <em>inference-compute</em>
budgets — benchmark on serving comparable to what you deploy.</li>
</ul>
<h2>Sources</h2>
<ul>
<li><a href="https://huggingface.co/abacusai/Smaug-Flash" target="_blank" rel="noopener">Model card</a> ·
<a href="https://huggingface.co/abacusai/Smaug-Flash/tree/main/encoding" target="_blank" rel="noopener">encoding/</a> ·
<a href="https://huggingface.co/abacusai/Smaug-Flash/tree/main/inference" target="_blank" rel="noopener">inference/</a></li>
<li><a href="https://huggingface.co/abacusai/Smaug-Flash/blob/main/figures/smaug_flash_vs_base_bars.png" target="_blank" rel="noopener">figures/smaug_flash_vs_base_bars.png</a> ·
<a href="https://huggingface.co/abacusai/Smaug-Flash/blob/main/figures/smaug_flash_livebench_categories.png" target="_blank" rel="noopener">figures/smaug_flash_livebench_categories.png</a></li>
</ul>
</section>
<!-- ===================================================== Benchmarks -->
<section class="panel" id="tab-benchmarks">
<h2>Smaug-Flash vs its base</h2>
<p class="note">
Higher is better. Δ is shown in each axis label. Rows marked <code>*</code> use the base model's
vendor-reported score for the identical public task set; unstarred rows are paired runs under the
same harness (card footnote).
</p>
<div class="chart" id="chart-bars"></div>
<div class="table-wrap">
<table id="bench-table">
<thead><tr><th>Benchmark</th><th>Smaug-Flash</th><th>DeepSeek-V4-Flash-0731</th><th>Δ</th></tr></thead>
<tbody></tbody>
</table>
</div>
<h2>LiveBench 2026-06-25 category profile</h2>
<p class="note">Scores 0–100; overall = mean of the seven category averages. Same harness, same
questions, paired against the base model.</p>
<div class="two-col">
<div class="chart" id="chart-radar"></div>
<div class="chart" id="chart-delta"></div>
</div>
<div class="table-wrap">
<table id="lb-table">
<thead><tr><th>Category</th><th>Smaug-Flash</th><th>Base</th><th>Δ</th></tr></thead>
<tbody></tbody>
</table>
</div>
<div class="callout">
<strong>How to read this.</strong> The averages hide where the gain comes from: agentic coding and
instruction following move a lot, while reasoning, data analysis and language are flat-to-slightly-down.
That is the expected shape for an agentic-loop fine-tune of the attention factors only.
</div>
<div class="callout warn">
<strong>Two caveats, both from the card.</strong> (1) Measured on fresh 2026 task generations,
self-served — on timeout-bounded agentic benchmarks the <em>serving throughput is part of the
score</em>, so numbers are hardware-dependent. (2) The charts above are <strong>digitized from the
model card's figures</strong>; the originals below are authoritative. Where card prose and figure
disagree slightly (NL2Repo-Bench “+19” in prose vs “+19.1” in the figure), the figure value is used.
</div>
<h2>Original figures from the model card</h2>
<img class="figure" src="https://huggingface.co/abacusai/Smaug-Flash/resolve/main/figures/smaug_flash_vs_base_bars.png"
alt="Smaug-Flash vs DeepSeek-V4-Flash-0731 across agentic benchmarks" loading="lazy" />
<img class="figure" src="https://huggingface.co/abacusai/Smaug-Flash/resolve/main/figures/smaug_flash_livebench_categories.png"
alt="Smaug-Flash LiveBench category profile vs base" loading="lazy" />
</section>
<!-- ================================================== Prompt encoder -->
<section class="panel" id="tab-encoder">
<h2>No chat template — so here is the real one</h2>
<p>
<code>abacusai/Smaug-Flash</code> ships <strong>no Jinja chat template</strong>, so
<code>tokenizer.apply_chat_template()</code> is not an option. The repo's
<a href="https://huggingface.co/abacusai/Smaug-Flash/blob/main/encoding/encoding_dsv4.py" target="_blank" rel="noopener"><code>encoding/encoding_dsv4.py</code></a>
(MIT) is vendored into this Space <strong>byte-for-byte unmodified</strong> and executed below
inside Pyodide — real CPython, in your browser, no server round-trip.
</p>
<table class="kv">
<tr><td><code>reasoning_effort</code></td><td><code>low</code> = no preamble (0 chars) · <code>high</code> = 476 chars · <code>max</code> = 526 chars of injected instruction. <code>None</code> ⇒ <code>low</code>. Only effective in <code>thinking</code> mode, only on the first message.</td></tr>
<tr><td><code>thinking_mode</code></td><td><code>thinking</code> opens <code>&lt;think&gt;</code> after <code>&lt;|Assistant|&gt;</code>; <code>chat</code> closes it immediately with <code>&lt;/think&gt;</code> so the model answers directly.</td></tr>
<tr><td>Special tokens</td><td><code>&lt;|begin▁of▁sentence|&gt;</code> · <code>&lt;|User|&gt;</code> · <code>&lt;|Assistant|&gt;</code> · <code>&lt;|end▁of▁sentence|&gt;</code> · <code>&lt;think&gt;</code>/<code>&lt;/think&gt;</code> · <code>&lt;|latest_reminder|&gt;</code> · <code>|DSML|</code> for tool calls</td></tr>
</table>
<p class="note"><strong>Delimiter nuance worth knowing:</strong> the repo's <code>encoding_dsv4.py</code>
and <code>tokenizer.json</code> delimit special tokens with <code>U+FF5C</code>
(<code>|</code>, a fullwidth reverse solidus) rather than ASCII <code>|</code> — the two are
near-identical in most fonts, which is why hand-typed pipes silently fail to parse. Copy the tokens
from this page or from the repo, don't retype them; the decoder below repairs typed ASCII pipes
automatically and tells you when it did.</p>
<p class="status" id="py-status">Python runtime not loaded yet — click <em>Encode prompt</em> and it will
start (≈ 10 MB, one time).</p>
<div class="two-col wide-left">
<div>
<label class="field">messages — OpenAI format (<code>reasoning_content</code> and
<code>tool_calls</code> allowed, <code>tool</code> messages get merged into the preceding user turn)
<textarea id="messages-in" class="code" rows="16" spellcheck="false"></textarea>
</label>
<div class="row">
<button class="ghost" data-example="chat">example: plain chat</button>
<button class="ghost" data-example="agentic">example: agentic tool loop</button>
<button class="ghost" data-example="custom">example: custom instructions</button>
</div>
<div class="row controls">
<label class="field">thinking_mode
<select id="thinking-in">
<option value="thinking" selected>thinking</option>
<option value="chat">chat</option>
</select>
</label>
<label class="field">reasoning_effort
<select id="effort-in">
<option value="low">low</option>
<option value="high">high</option>
<option value="max" selected>max</option>
</select>
</label>
<label class="check">
<input type="checkbox" id="drop-in" checked />
drop_thinking (production behaviour)
</label>
</div>
<details>
<summary>Tool schemas (optional JSON)</summary>
<textarea id="tools-in" class="code" rows="10" spellcheck="false" placeholder="[]"></textarea>
<p class="note">Declaring tools switches the encoder's <code>drop_thinking</code> off by itself —
the model was trained to keep its reasoning across tool loops.</p>
</details>
<button id="encode-btn" class="primary">Encode prompt</button>
</div>
<div>
<label class="field">encoded prompt — this is exactly what you tokenize
<textarea id="prompt-out" class="code mono-out" rows="22" readonly spellcheck="false"></textarea>
</label>
<div id="encode-stats" class="stats"></div>
<div id="encode-notes" class="notes"></div>
</div>
</div>
<h2>Decode a completion back into OpenAI format</h2>
<p class="note">The parser only accepts well-formed output — the card is explicit that it does not
recover from malformed generations: the think block must be closed with <code>&lt;/think&gt;</code>
and the turn must end with <code>&lt;|end▁of▁sentence|&gt;</code>.</p>
<div class="two-col wide-left">
<div>
<label class="field">raw model completion
<textarea id="completion-in" class="code" rows="6" spellcheck="false">Let me add the two integers.</think>The answer is 4.<|end▁of▁sentence|></textarea>
</label>
<div class="row controls">
<label class="field">thinking_mode
<select id="decode-mode">
<option value="thinking" selected>thinking</option>
<option value="chat">chat</option>
</select>
</label>
<button id="decode-btn" class="primary">Parse completion</button>
</div>
</div>
<div>
<label class="field">parsed assistant message
<textarea id="parsed-out" class="code mono-out" rows="10" readonly spellcheck="false"></textarea>
</label>
<div id="decode-status" class="stats"></div>
</div>
</div>
</section>
<!-- ===================================================== Playground -->
<section class="panel" id="tab-playground">
<h2>Chat with Smaug-Flash — through an endpoint you own</h2>
<p>
This page is static: there is no backend, no GPU and no proxy. Your browser calls an
OpenAI-compatible endpoint <strong>directly</strong>. Serve the model with vLLM or SGLang (see
<a href="#" data-goto="deploy">Deploy &amp; size</a>) and paste the base URL below. Anything
OpenAI-compatible works — including a model on the HF router, for a plumbing check.
</p>
<div class="callout warn">
<strong>Two browser realities.</strong> (1) <em>CORS:</em> the endpoint must send
<code>Access-Control-Allow-Origin</code>; vLLM allows all origins by default, most hosted APIs do too,
but a bare <code>python -m http.server</code> does not. (2) <em>HTTPS:</em> this page is served over
HTTPS, so a plain <code>http://</code> endpoint will be blocked as mixed content — terminate TLS
(tunnel, reverse proxy) or open this page over <code>http://localhost</code> from the repo.
Your key lives only in this tab: it is sent to the endpoint you type and nowhere else.
</div>
<div class="two-col wide-right">
<div class="side">
<label class="field">preset
<select id="preset">
<option value="own" selected>Your own Smaug-Flash deployment (vLLM / SGLang)</option>
<option value="router">HF Inference Providers router (live sibling model)</option>
<option value="local">Ollama / LM Studio / llama.cpp (same machine)</option>
<option value="custom">Custom</option>
</select>
</label>
<label class="field">base URL<input id="base-url" type="text" spellcheck="false" /></label>
<label class="field">model<input id="model-name" type="text" spellcheck="false" /></label>
<label class="field">API key<input id="api-key" type="password" spellcheck="false" /></label>
<p class="note" id="preset-hint"></p>
<button id="test-btn" class="ghost">Test endpoint (GET /v1/models)</button>
<div id="test-out" class="stats"></div>
<details open>
<summary>Sampling &amp; reasoning</summary>
<label class="field">system prompt<textarea id="system-in" rows="3">You are Smaug-Flash, a rigorous agentic coding assistant. Verify your work before you finish.</textarea></label>
<div class="row controls">
<label class="field">thinking_mode
<select id="pg-mode"><option value="thinking" selected>thinking</option><option value="chat">chat</option></select>
</label>
<label class="field">reasoning_effort
<select id="pg-effort"><option value="low">low</option><option value="high">high</option><option value="max" selected>max</option></select>
</label>
</div>
<label class="check"><input type="checkbox" id="pg-send" checked /> send <code>reasoning_effort</code> / <code>thinking_mode</code> as request parameters (falls back automatically if the server rejects them)</label>
<label class="field">temperature (card: 1.0)<input id="pg-temp" type="range" min="0" max="2" step="0.05" value="1" /><span id="pg-temp-v" class="mono">1.00</span></label>
<label class="field">top_p (card: 0.95)<input id="pg-topp" type="range" min="0.05" max="1" step="0.01" value="0.95" /><span id="pg-topp-v" class="mono">0.95</span></label>
<label class="field">max_tokens (card: up to 384K for high/max)<input id="pg-maxtok" type="number" min="16" max="384000" step="256" value="8192" /></label>
</details>
</div>
<div class="chat-col">
<div id="chat-log" class="chat-log"><p class="note">Nothing here yet — set an endpoint, then send a message.</p></div>
<label class="field">message<textarea id="msg-in" rows="3" placeholder="Ask the model something, or paste a repo task…"></textarea></label>
<div class="row">
<button id="send-btn" class="primary">Send</button>
<button id="clear-btn" class="ghost">Clear</button>
<span id="stream-status" class="stats"></span>
</div>
<label class="field">live reasoning stream (<code>reasoning_content</code>, if the server emits it)
<textarea id="reasoning-out" class="code mono-out" rows="8" readonly spellcheck="false"></textarea>
</label>
</div>
</div>
</section>
<!-- =================================================== Deploy & size -->
<section class="panel" id="tab-deploy">
<h2>Serve the real thing</h2>
<p>Any stack that runs <code>deepseek-ai/DeepSeek-V4-Flash-0731</code> runs Smaug-Flash unmodified —
only the attention factors changed, so the FP4 expert kernels and the DSpark speculative path are
untouched.</p>
<h3>SGLang ≥ 0.5.15 (Blackwell) — the card's primary recipe</h3>
<div class="recipe"><button class="copy ghost" data-copy="sglang-cmd">copy</button><pre id="sglang-cmd"></pre></div>
<h3>vLLM (single 4×GB300 node)</h3>
<div class="recipe"><button class="copy ghost" data-copy="vllm-cmd">copy</button><pre id="vllm-cmd"></pre></div>
<h3>Reference transformer path (the repo's own <code>inference/</code> folder)</h3>
<div class="recipe"><button class="copy ghost" data-copy="torch-cmd">copy</button><pre id="torch-cmd"></pre></div>
<h3>Python client (no chat template, so encode first)</h3>
<div class="recipe"><button class="copy ghost" data-copy="py-cmd">copy</button><pre id="py-cmd"></pre></div>
<p class="note">Serving settings the card is explicit about: <code>reasoning_effort=max</code>,
<code>temperature=1.0</code>, <code>top_p=0.95</code>, fp8 KV cache, full 1 M context, and up to
<strong>384 K</strong> output tokens for high/max effort. Local weight conversion and interactive
demos live in the repo's <code>inference/</code> folder.</p>
<h2>Memory calculator: where would this even run?</h2>
<p class="note">Parameter split is computed from <code>config.json</code>: 43 layers × (256 routed + 1
shared) experts × 3 matrices × 4096 × 2048 = <strong>278.1 B</strong> expert parameters; the
remaining <strong>26.1 B</strong> are attention, embeddings and the DSpark module. Total
<strong>304.18 B</strong>, matching the Hub's safetensors metadata.</p>
<div class="row controls">
<label class="field">non-expert params<select id="attn-bits">
<option value="1" selected>fp8 (1 byte) — native</option>
<option value="2">bf16</option>
<option value="0.5">fp4/int4 packed (0.5 byte)</option>
</select></label>
<label class="field">expert params<select id="expert-bits">
<option value="0.5" selected>fp4/int4 packed (0.5 byte) — native</option>
<option value="1">fp8 (1 byte)</option>
<option value="2">bf16</option>
</select></label>
<label class="field">headroom % (KV cache, activations, overhead)
<input id="headroom" type="range" min="0" max="60" step="5" value="15" /><span id="headroom-v" class="mono">15</span>
</label>
</div>
<div class="two-col wide-left">
<div id="size-out" class="notes"></div>
<div class="chart" id="chart-size"></div>
</div>
<div class="two-col">
<div>
<h3>Does it fit anywhere on Spaces?</h3>
<div class="table-wrap"><table id="hw-table">
<thead><tr><th>Flavor</th><th>Hardware</th><th>VRAM</th><th>Price</th><th>vs budget</th></tr></thead>
<tbody></tbody>
</table></div>
</div>
<div>
<h3>Notes that change the answer</h3>
<ul class="notes">
<li>The official recipes use <code>tp=4</code> / <code>data-parallel-size 4</code> with expert
parallelism and the <code>flashinfer_mxfp4</code> / <code>deep_gemm_mega_moe</code> MoE backends —
the packed-FP4 experts need kernels that exist only on recent-GPU stacks (SGLang ≥ 0.5.15 on
Blackwell).</li>
<li>“Just quantize harder” is not a free win: the card warns that re-quantizing merged weights
degrades constrained instruction-following.</li>
<li>KV-cache size is deliberately excluded — MLA's compressed ratios in <code>config.json</code>
make a naive per-token number misleading; use the headroom slider instead.</li>
<li>Paid Spaces tiers bill <strong>per hour while the Space runs</strong>. ZeroGPU (this Space's
flavor) is still capped at 96 GB — far below the 154 GB native budget above.</li>
</ul>
<h3>Cheaper routes that work today</h3>
<ol class="notes">
<li><strong>Tunnel a deployment</strong> (cloudflared / ngrok / tailscale funnel) and paste the
public HTTPS URL into the Playground.</li>
<li><strong>Open this page locally</strong> (<code>python -m http.server</code> in the repo) and
talk to <code>http://127.0.0.1:8000/v1</code> with no TLS or CORS friction.</li>
<li><strong>Community GPU grant</strong> if you want the Space itself to serve it: open a
discussion titled <code>Apply for a GPU community grant: Personal project</code> on the Space,
explaining it is a public open-source demo of an MIT-licensed 304 B model that cannot fit in
ZeroGPU's 96 GB. Approval typically takes days; hardware is then attached automatically.</li>
</ol>
</div>
</div>
</section>
</main>
<footer>
<p>
Built from the <a href="https://huggingface.co/abacusai/Smaug-Flash" target="_blank" rel="noopener">Smaug-Flash</a>
model card and repository contents (<code>config.json</code>, <code>encoding/encoding_dsv4.py</code>,
<code>figures/</code>), MIT licensed; the encoder is vendored unmodified and runs in your browser.
Benchmark values digitized from the card's figures — the originals remain authoritative.
Not affiliated with Abacus.AI or DeepSeek-AI.
</p>
</footer>
<script src="app.js"></script>
</body>
</html>