Hugging Face's logo

Spaces:
wpferrell
/
README
Configuration error

App Files Community
README / README.md
wpferrell's picture
wpferrell
card: bigsmall 4.0.0 - autopilot + Ferrell Duo
84ad007 verified
preview code
|
Raw
History Blame Contribute Delete
4.68 kB

DRAFT — DO NOT UPLOAD. Strip this block before publishing. Publishes at V4-REL on Will's word (V4_DOCS_SPEC.md Deliverable 8/9). Every number is traced in V4_DOCS_REPORT.md appendix A (claims ledger). Supersedes _v315/PUBLIC_DRAFTS/hf_card_README.md (the "Coming in 3.15" block there is obsolete — 3.15 shipped 2026-06-11, and this card is the 4.0 story).

BigSmall — Lossless AI Model Compression

Lossless AI model compression — ~34% smaller with bit-identical weights; the autopilot profiles your machine, picks the highest fidelity that runs, and streams models bigger than your RAM.

pip install bigsmall          # CLI + compression/decompression
pip install bigsmall[torch]   # add this for model loading (from_pretrained)

Every model on this page is a real measurement: a 14 GB Mistral-7B is 8.9 GB as a .bs, a 29.5 GB Qwen2.5-14B-Instruct is 19.5 GB — and after decompression every weight is bit-for-bit identical to the original (md5 verified per tensor on decompress).

  • Not quantization. Nothing is rounded; the model's behaviour cannot change.
  • Not pruning, not approximation. The same idea as ZIP for text, tuned for the statistics of trained BF16 weights.
  • Fine-tunes ship as deltas. A >=7B official instruct tune stored against its public base measures 34–50% of the full model. Delta size is pair-dependent (measured range: under 1% for the best >=7B SFT pairs, up to ~61% for small-model full tunes) — the docs carry the full measured table.

Using a pre-compressed model 

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "wpferrell/phi-3.5-mini-instruct-bigsmall"
)

BigSmall decompresses transparently on load. Prefer the CLI? Download the repo and run bigsmall decompress model-00001-of-00002.bs -o model.safetensors.

New in 4.0 — the autopilot release

Three commands replace every settings decision:

bigsmall profile     # once: ~10s hardware probe (GPU read via NVML only, never touched)
bigsmall plan f.bs   # one sentence: what would run, where, how faithfully, how fast
bigsmall run f.bs    # do it

The rule is highest fidelity at usable speed, and the honesty rules are enforced by the test suite: anything below bit-exact is announced before it happens, never silently.

  • The Ferrell Duo (.bsd) — one file. Two models: the fast one and the real one. A lossy-INT4 fast member (load by reading ~21% of raw bytes) and the lossless residual that reconstructs the original bits; the bit-exact gate runs before the file may exist. Measured cost: ~2.9 pp of raw over lossless-only. To our knowledge (as of 2026-06-12) the only shipped LLM-weights format whose top tier is the bit-exact original — adjacent art (MatQuant, Any-Precision LLM, audio hybrid formats) credited in the repo's comparison page.
  • Streaming executor — run models bigger than your RAM with a bounded, promised, measured resident set. Receipt at 7B: Qwen2.5-7B (14.2 GB raw, 65.95% compressed) streamed forward bit-exact vs the fully-loaded model (identical logits sha256) in ~2.5 GB of hot state.
  • FP8-native lossless — a real fp8 release (Qwen3-0.6B-FP8) compressed to 0.829 of its fp8 weight bytes, 507/507 tensors bit-exact.
  • bigsmall xray — checkpoint forensics with a "looks untrained" trap that catches silently-randomized loads; works on bf16 and fp8.

Honest numbers

The lossless floor for trained BF16 weights is real and measured: across 4,143 weight matrices in 8 architectures the per-tensor entropy floor is flat (CV ~ 0), and trained mantissa/sign bits are coder-equivalent to matched random controls — training only writes the exponent. BigSmall codes at that wall. Head-to-head under the same accounting, BigSmall codes below DFloat11's bound on every layer type of every model measured (+0.45–0.55 pp model-level, +12–18 pp on norm scales) — see docs/dfloat11.md in the source repo; the full landscape (DFloat11, ZipNN, ZipServ, Unweight, quantization, and the rest) is one honest page at docs/comparison.md.

Links