Spaces:
Running
Running
| <html lang="en"> | |
| <head> | |
| <meta charset="utf-8"> | |
| <meta name="viewport" content="width=device-width, initial-scale=1"> | |
| <title>Does this pack keep what it claims?</title> | |
| <!-- The Space renders inside an iframe and huggingface.co sends | |
| x-frame-options: DENY, so any link without a target tries to load | |
| a refused page in the frame and reads as broken. Defaulting every | |
| link out of the frame fixes the one that was, and the next one | |
| nobody remembers to mark. --> | |
| <base target="_blank"> | |
| <style> | |
| :root { | |
| --bg: #fbfbfa; | |
| --panel: #ffffff; | |
| --ink: #1a1a19; | |
| --muted: #6b6b66; | |
| --line: #e4e3df; | |
| --accent: #3d5a80; | |
| --problem: #a03030; | |
| --problem-bg: #fdf3f2; | |
| --check: #8a6212; | |
| --check-bg: #fdf9ee; | |
| --clear: #2f6b45; | |
| --clear-bg: #f1f8f3; | |
| --mono: ui-monospace, "SF Mono", "Cascadia Mono", Menlo, monospace; | |
| } | |
| @media (prefers-color-scheme: dark) { | |
| :root { | |
| --bg: #16171a; | |
| --panel: #1d1f23; | |
| --ink: #e8e6e3; | |
| --muted: #9a978f; | |
| --line: #2e3137; | |
| --accent: #8ab0d9; | |
| --problem: #e08a86; | |
| --problem-bg: #2a1e1e; | |
| --check: #d9b76a; | |
| --check-bg: #29241a; | |
| --clear: #8fc9a6; | |
| --clear-bg: #1b2620; | |
| } | |
| } | |
| * { box-sizing: border-box; } | |
| body { | |
| margin: 0; | |
| background: var(--bg); | |
| color: var(--ink); | |
| font: 16px/1.65 ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif; | |
| -webkit-font-smoothing: antialiased; | |
| } | |
| .wrap { max-width: 50rem; margin: 0 auto; padding: 3.5rem 1.25rem 5rem; } | |
| /* The mark is white-on-transparent — made for a dark background, so | |
| invisible on the light one. It is 100% monochrome, checked pixel | |
| by pixel, so a straight inversion is clean rather than a | |
| colour-shifted approximation. */ | |
| .brand { display: inline-block; margin-bottom: 1.75rem; } | |
| .brand img { | |
| height: 1.5rem; width: auto; display: block; | |
| filter: invert(1); opacity: .78; | |
| } | |
| @media (prefers-color-scheme: dark) { | |
| .brand img { filter: none; opacity: .9; } | |
| } | |
| header h1 { | |
| font-size: clamp(1.6rem, 4.5vw, 2.2rem); | |
| line-height: 1.2; | |
| letter-spacing: -0.02em; | |
| margin: 0 0 .75rem; | |
| font-weight: 620; | |
| } | |
| header p { color: var(--muted); margin: 0 0 .5rem; max-width: 42rem; } | |
| form { | |
| display: flex; gap: .5rem; flex-wrap: wrap; | |
| margin: 2rem 0 .75rem; | |
| } | |
| input[type=text] { | |
| flex: 1 1 20rem; min-width: 0; | |
| padding: .7rem .85rem; | |
| border: 1px solid var(--line); border-radius: .5rem; | |
| background: var(--panel); color: var(--ink); | |
| font: inherit; font-family: var(--mono); font-size: .92rem; | |
| } | |
| input[type=text]:focus { | |
| outline: 2px solid var(--accent); outline-offset: -1px; border-color: transparent; | |
| } | |
| button { | |
| padding: .7rem 1.4rem; border: 0; border-radius: .5rem; | |
| background: var(--accent); color: #fff; | |
| font: inherit; font-weight: 560; cursor: pointer; | |
| } | |
| button:hover { filter: brightness(1.08); } | |
| button:disabled { opacity: .55; cursor: progress; } | |
| .examples { | |
| font-size: .86rem; color: var(--muted); margin-bottom: 2.5rem; | |
| line-height: 2; | |
| } | |
| .examples .group { display: inline-block; min-width: 5.5rem; } | |
| .examples button { | |
| background: none; color: var(--accent); padding: 0 .15rem; | |
| font-size: .86rem; font-family: var(--mono); text-decoration: underline; | |
| text-underline-offset: 2px; font-weight: 400; | |
| } | |
| .card { | |
| background: var(--panel); border: 1px solid var(--line); | |
| border-radius: .7rem; padding: 1.1rem 1.3rem; margin: .85rem 0; | |
| } | |
| .card h3 { | |
| margin: 0 0 .5rem; font-size: 1.02rem; font-weight: 600; | |
| display: flex; gap: .55rem; align-items: baseline; | |
| } | |
| .card p { margin: .5rem 0; } | |
| .card ul { margin: .5rem 0; padding-left: 1.1rem; } | |
| .card li { font-family: var(--mono); font-size: .84rem; margin: .15rem 0; } | |
| .problem { border-left: 3px solid var(--problem); background: var(--problem-bg); } | |
| .problem h3 { color: var(--problem); } | |
| .check { border-left: 3px solid var(--check); background: var(--check-bg); } | |
| .check h3 { color: var(--check); } | |
| .clear { border-left: 3px solid var(--clear); background: var(--clear-bg); } | |
| .clear h3 { color: var(--clear); } | |
| .meta { | |
| display: flex; flex-wrap: wrap; gap: .35rem 1.5rem; | |
| font-size: .86rem; color: var(--muted); | |
| padding-bottom: .9rem; margin-bottom: .3rem; | |
| border-bottom: 1px solid var(--line); | |
| } | |
| .meta code { color: var(--ink); } | |
| code { font-family: var(--mono); font-size: .86em; } | |
| h2 { | |
| font-size: 1.05rem; font-weight: 600; letter-spacing: -0.01em; | |
| margin: 3rem 0 .75rem; | |
| } | |
| table { border-collapse: collapse; width: 100%; font-size: .87rem; } | |
| th, td { | |
| text-align: left; padding: .45rem .7rem .45rem 0; | |
| border-bottom: 1px solid var(--line); vertical-align: top; | |
| } | |
| th { font-weight: 600; color: var(--muted); font-size: .8rem; | |
| text-transform: uppercase; letter-spacing: .04em; } | |
| td code { font-size: .82rem; } | |
| .scroll { overflow-x: auto; } | |
| footer { | |
| margin-top: 3.5rem; padding-top: 1.5rem; | |
| border-top: 1px solid var(--line); | |
| font-size: .87rem; color: var(--muted); | |
| } | |
| footer a { color: var(--accent); } | |
| .spin { color: var(--muted); font-size: .9rem; } | |
| ol.checks { padding-left: 1.25rem; } | |
| ol.checks li { margin: .4rem 0; } | |
| </style> | |
| </head> | |
| <body> | |
| <div class="wrap"> | |
| <header> | |
| <a class="brand" href="https://assert.gr" target="_blank" rel="noopener"> | |
| <img src="./assert-logo.png" alt="ASSERT" width="420" height="87"> | |
| </a> | |
| <h1>Does this pack keep what it claims?</h1> | |
| <p> | |
| Quantization can drop or disable part of a model without failing. The pack | |
| loads, serves, and answers correctly — while a component its card says it | |
| preserved is either gone, or present and ignored by the runtime. Nothing | |
| errors. Nothing in the logs says so. | |
| </p> | |
| <p> | |
| This reads a published repo's <strong>metadata only</strong>: safetensors | |
| headers and <code>config.json</code>, fetched over ranged requests, from | |
| your browser. No weights are downloaded, so a 27 GB pack costs a few | |
| kilobytes to check. | |
| </p> | |
| </header> | |
| <form id="form"> | |
| <input type="text" id="repo" value="aleada/Qwen3.6-27B-W4A16" | |
| placeholder="owner/model" autocomplete="off" spellcheck="false" | |
| aria-label="Model repo id"> | |
| <button type="submit" id="go">Check</button> | |
| </form> | |
| <div class="examples"> | |
| <span class="group">Our packs:</span> | |
| <button type="button" data-ex="aleada/Qwen3.6-27B-W4A16">Qwen3.6-27B</button> · | |
| <button type="button" data-ex="aleada/Gemma-4-12B-it-W4A16">Gemma-4-12B</button> · | |
| <button type="button" data-ex="aleada/Pixtral-12B-W4A16">Pixtral-12B</button> · | |
| <button type="button" data-ex="aleada/Llama-3.2-11B-Vision-Instruct-W4A16">Llama-3.2-Vision</button> | |
| <br> | |
| <span class="group">Elsewhere:</span> | |
| <button type="button" data-ex="groxaxo/Qwen3.6-27B-GPTQ-Pro-4bit">one missing its draft head</button> · | |
| <button type="button" data-ex="QuantTrio/Qwen3.6-27B-AWQ">a clean pack, another format</button> | |
| </div> | |
| <div id="out"></div> | |
| <h2>What it checks</h2> | |
| <ol class="checks"> | |
| <li> | |
| <strong>Does every exclusion entry name a real module?</strong> An entry | |
| naming something this architecture does not have protects nothing, and | |
| whatever it was meant to keep was quantized. The common case is a vision | |
| tower: Llama and Pixtral call it <code>vision_tower</code>, Qwen calls it | |
| <code>visual</code>, and the documented default names the former. | |
| </li> | |
| <li> | |
| <strong>Did any module of the source model fail to reach the pack?</strong> | |
| <code>from_pretrained</code> does not materialise tensors the AutoModel | |
| class has no slot for, so an auxiliary head living inside the checkpoint | |
| never reaches the quantizer and never reaches the artifact. | |
| </li> | |
| <li> | |
| <strong>Is anything at source precision without being declared?</strong> A | |
| runtime builds those quantized, looks for a packed weight, finds a plain | |
| one, and skips it — leaving the module randomly initialised. | |
| </li> | |
| </ol> | |
| <p> | |
| The first two are observations. The third is a <strong>prediction</strong>, | |
| and says so where it appears. | |
| </p> | |
| <h2>Why the rules are per format</h2> | |
| <p> | |
| Every format declares exclusions differently, and reading one convention into | |
| another is how a checker invents verdicts. A bare <code>mtp</code> covers a | |
| whole head under AWQ's substring rule and covers <em>nothing</em> under | |
| compressed-tensors' exact-match rule. | |
| </p> | |
| <div class="scroll"> | |
| <table> | |
| <thead><tr><th>Format</th><th>Field</th><th>How an entry matches</th></tr></thead> | |
| <tbody> | |
| <tr><td>compressed-tensors</td><td><code>ignore</code></td><td>exact equality, or a <code>re:</code> regex</td></tr> | |
| <tr><td>awq</td><td><code>modules_to_not_convert</code></td><td>plain substring</td></tr> | |
| <tr><td>gptq</td><td><code>dynamic</code></td><td>ordered, first match wins; <code>-:</code> excludes</td></tr> | |
| <tr><td>bitsandbytes</td><td><code>llm_int8_skip_modules</code></td><td>exact path component, or a cumulative prefix</td></tr> | |
| <tr><td>modelopt</td><td><code>exclude_modules</code></td><td>exact, substring, or an fnmatch glob</td></tr> | |
| <tr><td>auto-round</td><td><code>extra_config</code></td><td>per-module <code>bits ≥ 16</code>, or a block allowlist</td></tr> | |
| </tbody> | |
| </table> | |
| </div> | |
| <p> | |
| Every regex above resolves through Python's <code>re.match</code>, which | |
| anchors at the start: <code>re:layers.0</code> does <em>not</em> match | |
| <code>mtp.layers.0…</code>. These were read out of vLLM's source, not from | |
| documentation. A format not on the list is reported as <strong>not | |
| checked</strong> rather than assumed healthy — a checker that cannot tell | |
| "healthy" from "not checked" is not a checker. | |
| </p> | |
| <footer> | |
| <p> | |
| <strong>A clean result is not a quality measurement.</strong> It means the | |
| three checks found nothing. This never loads the model and never runs it. A | |
| pack can pass here and still be a poor quantization. | |
| </p> | |
| <p> | |
| Rules extracted from vLLM <code>0.23.1rc1.dev552+g4559c43a9</code> on | |
| 2026-08-11; they are not a stable public API upstream. Check 3 tolerates | |
| the wrapper-level difference between the names written into an exclusion | |
| list and the names tensors are stored under, because vLLM resolves that with | |
| a per-architecture mapper this cannot replicate from metadata — so it errs | |
| toward "declared", missing some real cases rather than accusing correct | |
| packs. Gated and private repos cannot be read. | |
| </p> | |
| <p> | |
| Built from the tooling behind the | |
| <a href="https://huggingface.co/aleada" rel="noopener">aleada</a> W4A16 packs. | |
| </p> | |
| </footer> | |
| </div> | |
| <script type="module" src="./app.js"></script> | |
| </body> | |
| </html> | |