README.md

---
title: Does this pack keep what it claims?
emoji: 🔎
colorFrom: indigo
colorTo: gray
sdk: static
app_file: index.html
pinned: false
license: apache-2.0
short_description: Check a published quantized model against its own config
---

# Quantized pack integrity check

Quantization can drop or disable part of a model without failing. The
pack loads, serves, and answers correctly — while a component its card
says it preserved is either absent or present-and-ignored by the
runtime. Nothing errors, and nothing in the logs says so.

Everything runs **in the visitor's browser**. There is no backend and
no token: it reads safetensors headers and `config.json` over ranged
requests, exactly what any visitor could read themselves. A 27 GB pack
costs a few kilobytes to inspect.

## What it checks

1. **Does every exclusion entry name a real module?** An entry naming
   something the architecture does not have protects nothing. The
   common case is a vision tower: Llama and Pixtral call it
   `vision_tower`, Qwen calls it `visual`, and the documented default
   names the former.
2. **Did any module of the source model fail to reach the pack?**
   `from_pretrained` does not materialise tensors the AutoModel class
   has no slot for, so an auxiliary head living inside the checkpoint
   never reaches the quantizer and never reaches the artifact.
3. **Is anything at source precision without being declared?** A
   runtime builds those quantized, looks for a packed weight, finds a
   plain one, and skips it.

The first two are observations. The third is a prediction, and says so.

## Two implementations, kept in step mechanically

The rules exist twice: in `../pack_check.py` for the bulk scanner, and
in `pack_check.js` here because a static Space has no Python. That is a
real risk — not that the port is wrong today, but that the two drift
later and only one of them is ever reviewed.

So the agreement is enforced rather than trusted:

```bash
python gen_convention_golden.py   # Python answers the shared cases
node parity.mjs                   # JavaScript must reproduce them
```

`conventions.cases.json` holds inputs only. The expectations are
generated from Python, never typed by hand — a hand-written
expectation records what someone believed on the day. Change a rule on
either side, regenerate, and the port fails until it agrees. Verified
that it can fail: breaking the regex anchoring (`re.match` semantics
to `search`) is caught immediately.

## Why the rules are per format

| format | field | how an entry matches |
|---|---|---|
| compressed-tensors | `ignore` | exact equality, or a `re:` regex |
| awq | `modules_to_not_convert` | plain substring |
| gptq | `dynamic` | ordered, first match wins; `-:` excludes |
| bitsandbytes | `llm_int8_skip_modules` | exact path component, or a cumulative prefix |
| modelopt | `exclude_modules` | exact, substring, or an fnmatch glob |
| auto-round | `extra_config` | per-module `bits >= 16`, or a block allowlist |

A bare `mtp` covers a whole head under AWQ's substring rule and covers
**nothing** under compressed-tensors' exact-match rule. Every regex
resolves through Python's `re.match`, which anchors at the start:
`re:layers.0` does not match `mtp.layers.0.…`.

These were read out of vLLM's source, not from documentation. A format
that is not on the list reports **not checked** rather than OK — a
checker that cannot tell "healthy" from "not checked" is not a checker.

## Limits

- A clean result is not a quality measurement. This never loads the
  model and never runs it. A pack can pass here and still be a poor
  quantization.
- Rules extracted from vLLM `0.23.1rc1.dev552+g4559c43a9` on
  2026-08-11. They are not a stable public API upstream.
- Check 3 tolerates the wrapper-level difference between the names
  written into an exclusion list and the names tensors are stored
  under, because vLLM resolves that with a per-architecture mapper this
  cannot replicate from metadata. It errs toward "declared" — missing
  some real cases rather than accusing correct packs.
- Gated and private repos cannot be read.

## Deploying

```bash
HF_TOKEN=... ./deploy.sh aleada/pack-integrity-check
```

Static Spaces are free; Gradio and Docker Spaces require PRO. A Gradio
version of the same checker lives in [`../space/`](../space/) for if
that changes — it shares `pack_check.py` directly and needs no port.