Spaces:

aleada
/

pack-integrity-check

Running

File size: 11,093 Bytes

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Does this pack keep what it claims?</title>
<!-- The Space renders inside an iframe and huggingface.co sends

     x-frame-options: DENY, so any link without a target tries to load

     a refused page in the frame and reads as broken. Defaulting every

     link out of the frame fixes the one that was, and the next one

     nobody remembers to mark. -->
<base target="_blank">
<style>

  :root {

    --bg: #fbfbfa;

    --panel: #ffffff;

    --ink: #1a1a19;

    --muted: #6b6b66;

    --line: #e4e3df;

    --accent: #3d5a80;

    --problem: #a03030;

    --problem-bg: #fdf3f2;

    --check: #8a6212;

    --check-bg: #fdf9ee;

    --clear: #2f6b45;

    --clear-bg: #f1f8f3;

    --mono: ui-monospace, "SF Mono", "Cascadia Mono", Menlo, monospace;

  }

  @media (prefers-color-scheme: dark) {

    :root {

      --bg: #16171a;

      --panel: #1d1f23;

      --ink: #e8e6e3;

      --muted: #9a978f;

      --line: #2e3137;

      --accent: #8ab0d9;

      --problem: #e08a86;

      --problem-bg: #2a1e1e;

      --check: #d9b76a;

      --check-bg: #29241a;

      --clear: #8fc9a6;

      --clear-bg: #1b2620;

    }

  }



  * { box-sizing: border-box; }

  body {

    margin: 0;

    background: var(--bg);

    color: var(--ink);

    font: 16px/1.65 ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif;

    -webkit-font-smoothing: antialiased;

  }

  .wrap { max-width: 50rem; margin: 0 auto; padding: 3.5rem 1.25rem 5rem; }



  /* The mark is white-on-transparent — made for a dark background, so

     invisible on the light one. It is 100% monochrome, checked pixel

     by pixel, so a straight inversion is clean rather than a

     colour-shifted approximation. */

  .brand { display: inline-block; margin-bottom: 1.75rem; }

  .brand img {

    height: 1.5rem; width: auto; display: block;

    filter: invert(1); opacity: .78;

  }

  @media (prefers-color-scheme: dark) {

    .brand img { filter: none; opacity: .9; }

  }



  header h1 {

    font-size: clamp(1.6rem, 4.5vw, 2.2rem);

    line-height: 1.2;

    letter-spacing: -0.02em;

    margin: 0 0 .75rem;

    font-weight: 620;

  }

  header p { color: var(--muted); margin: 0 0 .5rem; max-width: 42rem; }



  form {

    display: flex; gap: .5rem; flex-wrap: wrap;

    margin: 2rem 0 .75rem;

  }

  input[type=text] {

    flex: 1 1 20rem; min-width: 0;

    padding: .7rem .85rem;

    border: 1px solid var(--line); border-radius: .5rem;

    background: var(--panel); color: var(--ink);

    font: inherit; font-family: var(--mono); font-size: .92rem;

  }

  input[type=text]:focus {

    outline: 2px solid var(--accent); outline-offset: -1px; border-color: transparent;

  }

  button {

    padding: .7rem 1.4rem; border: 0; border-radius: .5rem;

    background: var(--accent); color: #fff;

    font: inherit; font-weight: 560; cursor: pointer;

  }

  button:hover { filter: brightness(1.08); }

  button:disabled { opacity: .55; cursor: progress; }



  .examples {

    font-size: .86rem; color: var(--muted); margin-bottom: 2.5rem;

    line-height: 2;

  }

  .examples .group { display: inline-block; min-width: 5.5rem; }

  .examples button {

    background: none; color: var(--accent); padding: 0 .15rem;

    font-size: .86rem; font-family: var(--mono); text-decoration: underline;

    text-underline-offset: 2px; font-weight: 400;

  }



  .card {

    background: var(--panel); border: 1px solid var(--line);

    border-radius: .7rem; padding: 1.1rem 1.3rem; margin: .85rem 0;

  }

  .card h3 {

    margin: 0 0 .5rem; font-size: 1.02rem; font-weight: 600;

    display: flex; gap: .55rem; align-items: baseline;

  }

  .card p { margin: .5rem 0; }

  .card ul { margin: .5rem 0; padding-left: 1.1rem; }

  .card li { font-family: var(--mono); font-size: .84rem; margin: .15rem 0; }



  .problem { border-left: 3px solid var(--problem); background: var(--problem-bg); }

  .problem h3 { color: var(--problem); }

  .check   { border-left: 3px solid var(--check);   background: var(--check-bg); }

  .check h3 { color: var(--check); }

  .clear   { border-left: 3px solid var(--clear);   background: var(--clear-bg); }

  .clear h3 { color: var(--clear); }



  .meta {

    display: flex; flex-wrap: wrap; gap: .35rem 1.5rem;

    font-size: .86rem; color: var(--muted);

    padding-bottom: .9rem; margin-bottom: .3rem;

    border-bottom: 1px solid var(--line);

  }

  .meta code { color: var(--ink); }

  code { font-family: var(--mono); font-size: .86em; }



  h2 {

    font-size: 1.05rem; font-weight: 600; letter-spacing: -0.01em;

    margin: 3rem 0 .75rem;

  }

  table { border-collapse: collapse; width: 100%; font-size: .87rem; }

  th, td {

    text-align: left; padding: .45rem .7rem .45rem 0;

    border-bottom: 1px solid var(--line); vertical-align: top;

  }

  th { font-weight: 600; color: var(--muted); font-size: .8rem;

       text-transform: uppercase; letter-spacing: .04em; }

  td code { font-size: .82rem; }

  .scroll { overflow-x: auto; }



  footer {

    margin-top: 3.5rem; padding-top: 1.5rem;

    border-top: 1px solid var(--line);

    font-size: .87rem; color: var(--muted);

  }

  footer a { color: var(--accent); }

  .spin { color: var(--muted); font-size: .9rem; }

  ol.checks { padding-left: 1.25rem; }

  ol.checks li { margin: .4rem 0; }

</style>
</head>
<body>
<div class="wrap">

<header>
  <a class="brand" href="https://assert.gr" target="_blank" rel="noopener">
    <img src="./assert-logo.png" alt="ASSERT" width="420" height="87">
  </a>
  <h1>Does this pack keep what it claims?</h1>
  <p>
    Quantization can drop or disable part of a model without failing. The pack
    loads, serves, and answers correctly — while a component its card says it
    preserved is either gone, or present and ignored by the runtime. Nothing
    errors. Nothing in the logs says so.
  </p>
  <p>
    This reads a published repo's <strong>metadata only</strong>: safetensors
    headers and <code>config.json</code>, fetched over ranged requests, from
    your browser. No weights are downloaded, so a 27&nbsp;GB pack costs a few
    kilobytes to check.
  </p>
</header>

<form id="form">
  <input type="text" id="repo" value="aleada/Qwen3.6-27B-W4A16"

         placeholder="owner/model" autocomplete="off" spellcheck="false"

         aria-label="Model repo id">
  <button type="submit" id="go">Check</button>
</form>
<div class="examples">
  <span class="group">Our packs:</span>
  <button type="button" data-ex="aleada/Qwen3.6-27B-W4A16">Qwen3.6-27B</button> ·
  <button type="button" data-ex="aleada/Gemma-4-12B-it-W4A16">Gemma-4-12B</button> ·
  <button type="button" data-ex="aleada/Pixtral-12B-W4A16">Pixtral-12B</button> ·
  <button type="button" data-ex="aleada/Llama-3.2-11B-Vision-Instruct-W4A16">Llama-3.2-Vision</button>
  <br>
  <span class="group">Elsewhere:</span>
  <button type="button" data-ex="groxaxo/Qwen3.6-27B-GPTQ-Pro-4bit">one missing its draft head</button> ·
  <button type="button" data-ex="QuantTrio/Qwen3.6-27B-AWQ">a clean pack, another format</button>
</div>

<div id="out"></div>

<h2>What it checks</h2>
<ol class="checks">
  <li>
    <strong>Does every exclusion entry name a real module?</strong> An entry
    naming something this architecture does not have protects nothing, and
    whatever it was meant to keep was quantized. The common case is a vision
    tower: Llama and Pixtral call it <code>vision_tower</code>, Qwen calls it
    <code>visual</code>, and the documented default names the former.
  </li>
  <li>
    <strong>Did any module of the source model fail to reach the pack?</strong>
    <code>from_pretrained</code> does not materialise tensors the AutoModel
    class has no slot for, so an auxiliary head living inside the checkpoint
    never reaches the quantizer and never reaches the artifact.
  </li>
  <li>
    <strong>Is anything at source precision without being declared?</strong> A
    runtime builds those quantized, looks for a packed weight, finds a plain
    one, and skips it — leaving the module randomly initialised.
  </li>
</ol>
<p>
  The first two are observations. The third is a <strong>prediction</strong>,
  and says so where it appears.
</p>

<h2>Why the rules are per format</h2>
<p>
  Every format declares exclusions differently, and reading one convention into
  another is how a checker invents verdicts. A bare <code>mtp</code> covers a
  whole head under AWQ's substring rule and covers <em>nothing</em> under
  compressed-tensors' exact-match rule.
</p>
<div class="scroll">
<table>
  <thead><tr><th>Format</th><th>Field</th><th>How an entry matches</th></tr></thead>
  <tbody>
    <tr><td>compressed-tensors</td><td><code>ignore</code></td><td>exact equality, or a <code>re:</code> regex</td></tr>
    <tr><td>awq</td><td><code>modules_to_not_convert</code></td><td>plain substring</td></tr>
    <tr><td>gptq</td><td><code>dynamic</code></td><td>ordered, first match wins; <code>-:</code> excludes</td></tr>
    <tr><td>bitsandbytes</td><td><code>llm_int8_skip_modules</code></td><td>exact path component, or a cumulative prefix</td></tr>
    <tr><td>modelopt</td><td><code>exclude_modules</code></td><td>exact, substring, or an fnmatch glob</td></tr>
    <tr><td>auto-round</td><td><code>extra_config</code></td><td>per-module <code>bits&nbsp;≥&nbsp;16</code>, or a block allowlist</td></tr>
  </tbody>
</table>
</div>
<p>
  Every regex above resolves through Python's <code>re.match</code>, which
  anchors at the start: <code>re:layers.0</code> does <em>not</em> match
  <code>mtp.layers.0…</code>. These were read out of vLLM's source, not from
  documentation. A format not on the list is reported as <strong>not
  checked</strong> rather than assumed healthy — a checker that cannot tell
  "healthy" from "not checked" is not a checker.
</p>

<footer>
  <p>
    <strong>A clean result is not a quality measurement.</strong> It means the
    three checks found nothing. This never loads the model and never runs it. A
    pack can pass here and still be a poor quantization.
  </p>
  <p>
    Rules extracted from vLLM <code>0.23.1rc1.dev552+g4559c43a9</code> on
    2026-08-11; they are not a stable public API upstream. Check&nbsp;3 tolerates
    the wrapper-level difference between the names written into an exclusion
    list and the names tensors are stored under, because vLLM resolves that with
    a per-architecture mapper this cannot replicate from metadata — so it errs
    toward "declared", missing some real cases rather than accusing correct
    packs. Gated and private repos cannot be read.
  </p>
  <p>
    Built from the tooling behind the
    <a href="https://huggingface.co/aleada" rel="noopener">aleada</a> W4A16 packs.
  </p>
</footer>

</div>
<script type="module" src="./app.js"></script>
</body>
</html>