docs/profiled-bounds-implementation-plan.md

Profiled Bounds Implementation Plan

Purpose

Local Frontier should not compute serious model bounds from scraped Hugging Face metadata alone. The long-term system should be evidence-backed, reproducible, and fail closed.

The target architecture is:

scraped model catalog
  -> discovery and display only

audited profile registry
  -> self-contained model profiles

pure bounds engine
  -> hardware + model profile + workload settings
  -> derived batch, ceilings, statuses, and trace

The core rule is simple: no audited self-contained model profile means no serious tok/s bound.

Source Contract

This plan follows the bounds contract in:

https://github.com/osolmaz/onurclaw/blob/main/docs/2026-06-30-local-frontier-model-bounds.md

The engine must implement the adapter contract from that note:

Adapter(M) = [
  W_resident,
  W_batch(b),
  K_alloc(L_alloc),
  K_read(L_read),
  rho
]

The calculator must derive concurrency. It must not expose batch/concurrency as a primary user-controlled input.

Core Concepts

Model Profile

A model profile is one self-contained file for one concrete Hugging Face repo. It embeds all model data needed by the bounds engine:

• repo id
• profile status
• architecture data
• weight traffic adapter
• KV/state traffic adapter
• weight precision
• KV store precision
• KV read precision
• quantization or packaging assumptions
• base-model or artifact relation evidence, if any
• review status

A quantized artifact gets its own model profile because serving precision is part of the runtime input. A fine-tune gets its own model profile even when its embedded architecture data matches the base model.

Embedded Reuse

Model profiles are denormalized production artifacts. They do not point to separate runtime architecture or serving files.

Reuse happens through:

• shared adapter kinds and formulas
• schema definitions
• audit packet generation
• reviewer tooling that copies known-good architecture or serving blocks into many self-contained model files

Coverage rule:

• every scraped model repo must eventually have one model profile file
• a model profile may reuse copied architecture or serving blocks when evidence supports that reuse
• a model profile may mark the repo unsupported with a concrete reason
• production code must never infer a missing model profile from repo name, model name, parameter count, or quantization label
• shared formulas reduce duplicated logic; they do not replace per-repo files

Current registry sizing target from the 2026-07-01 scrape:

• 654 self-contained model profile files, one per scraped model row
• about 324 reusable architecture data shapes after exact structural deduplication, copied into model profiles as needed
• about 29 reusable serving data shapes after precision/representation deduplication, copied into model profiles as needed
• default workload settings live in app configuration, not in profile files

These counts are implementation targets for the current scrape, not schema constants. Validation should report drift when the scraped catalog changes.

Workload Settings

Workload settings are user or app configuration, not saved profile files:

• reserved context, L_alloc
• active read context, L_read
• minimum useful per-session rate, r_star
• decode policy, initially ordinary decode with rho = 1

The app should keep default workload settings in code or normal app config. Users may change them through the compare UI. They do not belong under profiles/.

Speculative decode is a future workload setting. It can allow rho > 1, but only with explicit draft/verifier traffic. Bounds Engine v1 should reject rho > 1 rather than treating raw rho as a free multiplier.

Adapter Axes

The profile system should be compositional.

model profile = architecture block + serving block + evidence + review/status
architecture block = weight traffic adapter + KV/state traffic adapter
serving block = precision and runtime representation
workload settings = context lengths, floor, decode policy

Weight Traffic Adapters

Initial supported weight adapters:

1. dense

   • all weights are resident
   • all weights are swept each decode step
   • W_resident = P_total * weight_bytes
   • W_batch(b) = W_resident

2. dense_resident_swept

   • all stored weights are resident
   • only the audited swept subset is charged each decode step
   • intended for packages such as multimodal checkpoints where a vision tower is resident but not swept for each generated text token
   • W_resident = P_resident * weight_bytes
   • W_batch(b) = P_swept * weight_bytes
   • requires evidence for the stored resident parameter count and swept parameter count

3. moe_distinct_experts

   • all experts are resident
   • per-batch weight traffic grows with expected distinct routed experts
   • requires total params, active params, routed expert count, routed experts per token, and optional shared-expert documentation

4. moe_distinct_experts_exact

   • all stored weights are resident
   • per-batch routed expert traffic grows with expected distinct routed experts
   • fixed and per-expert traffic are provided directly in GB
   • intended for audited mixed-dtype MoE checkpoints where a single weight_bytes_per_param would hide side-tensor overhead

The list is not final. Future adapters can include nonuniform MoE, shared-expert variants, mixture-of-depth, early-exit, external-memory, or other structures. Unsupported structures must remain unsupported until a new audited adapter exists.

KV / State Traffic Adapters

Initial supported KV/state adapters:

1. full_context

   • KV allocation and read traffic scale linearly with context length

2. layered_kv

   • combines components such as full-context layers and sliding-window layers
   • preferred representation for hybrid local/global attention

3. recurrent_state

   • fixed state instead of full KV growth

4. compressed_state

   • latent, sparse, compressed, or other non-full-KV state models

The durable abstraction should be a sum of KV components, not a growing list of one-off family enums.

Example:

{
  "kind": "layered_kv",
  "components": [
    {
      "kind": "full_context",
      "layers": 10,
      "kv_heads": 8,
      "head_dim": 256
    },
    {
      "kind": "sliding_window",
      "layers": 20,
      "kv_heads": 8,
      "head_dim": 256,
      "window_tokens": 4096
    }
  ]
}

Precision / Serving Blocks

Initial serving block families:

• BF16 or FP16 weights, BF16 or FP16 KV
• FP8 weights, BF16 or FP16 KV
• NVFP4 or MXFP4 weights, BF16 or FP16 KV
• INT8 weights, BF16 or FP16 KV
• INT4, Q4, AWQ, GPTQ, GGUF, or MLX weights with explicit KV precision
• Q2-style mixed profiles where evidence supports the assumption

Weight precision and KV precision must be separate. NVFP4 weights do not imply NVFP4 KV.

Detailed Profile Specifications

This section defines the production profile shapes in enough detail to write schemas and validators. It is intentionally stricter than the scrape metadata. Scrape data can suggest a candidate profile, but profile files are the audited contract.

Profile Registry Files

The registry should be plain files, not generated page routes:

profiles/
  models/
    google--gemma-4-26b-a4b-it.json
    redhatai--gemma-4-26b-a4b-it-nvfp4.json

File rules:

• file stems must match id
• ids must be lowercase and URL-safe
• model profile ids are derived from Hugging Face repo ids by lowercasing, replacing / with --, replacing other non-alphanumeric runs with -, and trimming leading or trailing separators
• model profiles must keep the original Hugging Face repo id in repo
• model profile files must be self-contained: no runtime references to external architecture or serving profile files
• generated static pages must not add index.html

Shared Profile Envelope

Every persisted model profile should share a common metadata envelope:

{
  "id": "redhatai--gemma-4-26b-a4b-it-nvfp4",
  "version": "1.0.0",
  "schema_version": "1.0.0",
  "status": "audited",
  "title": "RedHatAI Gemma 4 26B A4B NVFP4",
  "summary": "Self-contained model profile for memory-side bounds.",
  "evidence": [
    {
      "label": "Hugging Face model card",
      "url": "https://huggingface.co/RedHatAI/gemma-4-26B-A4B-it-NVFP4",
      "source_type": "model_card",
      "supports": [
        "repo",
        "serving"
      ]
    }
  ],
  "review": {
    "reviewed_by": "osolmaz",
    "reviewed_at": "2026-07-01",
    "notes": "Initial profile seeded from the model-bounds note."
  }
}

Common field rules:

• id must be stable, lowercase, URL-safe, and never derived from a transient artifact name when a base-family id exists.
• version changes when profile semantics change.
• schema_version changes when the schema changes.
• status controls whether the profile can produce production bounds.
• evidence must be present for audited model profiles.
• review must be present for audited model profiles.
• bounds results are output snapshots, not input profiles; they use engine_version plus a result schema_version.

Evidence Records

Evidence must be structured so audits can be repeated.

{
  "label": "Gemma 4 config",
  "url": "https://huggingface.co/google/gemma-4-26B-A4B-it/raw/main/config.json",
  "source_type": "config",
  "supports": [
    "layers",
    "kv_heads",
    "head_dim",
    "attention_pattern"
  ],
  "notes": "Used to verify the KV adapter fields."
}

Initial source_type values:

• model_card
• config
• paper
• vendor_doc
• release_notes
• benchmark_report
• manual_review
• derived_calculation

Evidence should state which profile fields it supports. A profile with evidence links but no field-level support is not audited.

Model Profile

Model profiles are the only production model artifact. Each model profile embeds architecture and serving data directly.

{
  "id": "qwen--qwen3-0-6b",
  "version": "1.0.0",
  "schema_version": "1.0.0",
  "status": "audited",
  "repo": "Qwen/Qwen3-0.6B",
  "model_family": "qwen3",
  "base_model_proof": {
    "base_model": "Qwen/Qwen3-0.6B-Base",
    "relation": "finetune",
    "source": "cardData.base_model",
    "config_compatible": true
  },
  "architecture": {
    "canonical_architecture_id": "qwen3-0-6b",
    "weight_adapter": {
      "kind": "dense",
      "total_params_b": 0.7516,
      "parameter_scope": "derived_from_safetensors_total"
    },
    "kv_adapter": {
      "kind": "full_context",
      "layers": 28,
      "kv_heads": 8,
      "head_dim": 128
    },
    "max_context_tokens": 40960
  },
  "serving": {
    "weight_format": "bf16",
    "weight_bytes_per_param": 2,
    "kv_store_format": "bf16",
    "kv_store_bytes_per_scalar": 2,
    "kv_read_format": "bf16",
    "kv_read_bytes_per_scalar": 2
  },
  "evidence": [
    {
      "label": "Qwen3 0.6B config",
      "url": "https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json",
      "source_type": "config",
      "supports": [
        "weight_adapter",
        "kv_adapter",
        "max_context_tokens"
      ]
    }
  ],
  "review": {
    "reviewed_by": "osolmaz",
    "reviewed_at": "2026-07-01",
    "notes": "Initial audited profile."
  }
}

Rules:

• Model profiles must include repo packaging and serving details because they are the runtime artifact.
• Fine-tuned instruction repos can copy architecture data from a base model only when the model profile records proof that the architecture did not change.
• Quantized artifacts can copy architecture data from the source model only when the model profile records proof that quantization did not change architecture.
• If an artifact changes attention, experts, pruning, routing, depth, or state, its model profile must embed different architecture data.
• Runtime code must not chase external architecture_profile or serving_profile references. Any such references are invalid in production model profiles.

Weight Adapter: dense

Use for models where every decode step touches all model weights.

{
  "kind": "dense",
  "total_params_b": 8.03,
  "parameter_scope": "total_including_embeddings",
  "formula": "W_resident = W_batch = total_params_b * weight_bytes_per_param"
}

Required fields:

• kind
• total_params_b
• parameter_scope

Optional fields:

• embedding_params_b
• non_embedding_params_b
• notes

Allowed parameter_scope values:

• total_including_embeddings
• total_excluding_embeddings
• marketed_parameter_count
• derived_from_safetensors_total

Formula:

W_resident = total_params_b * weight_bytes_per_param
W_batch(b) = W_resident

Audit requirements:

• total parameter count source
• whether embedding/head parameters are included
• explanation if parameter count is rounded or marketed

Weight Adapter: dense_resident_swept

Use for dense decoder packages where all stored weights must be resident, but only a verified subset is swept for each generated text token. This covers multimodal repos with resident vision towers and dense language decoders.

{
  "kind": "dense_resident_swept",
  "resident_params_b": 8.292166656,
  "swept_params_b": 7.615616512,
  "auxiliary_resident_params_b": 0.676550144,
  "resident_weight_gb": 16.584333312,
  "swept_weight_gb": 15.231233024,
  "resident_parameter_scope": "safetensors_header_stored_bf16",
  "swept_parameter_scope": "language_model_plus_lm_head_safetensors_headers",
  "auxiliary_scope": "visual tower resident during multimodal serving"
}

Required fields:

• kind
• resident_params_b
• swept_params_b
• resident_parameter_scope
• swept_parameter_scope

Optional fields:

• auxiliary_resident_params_b
• resident_weight_gb
• swept_weight_gb
• auxiliary_resident_weight_gb
• auxiliary_scope
• notes

Formula:

W_resident = resident_params_b * weight_bytes_per_param
W_batch(b) = swept_params_b * weight_bytes_per_param

If exact mixed-dtype byte totals are present:

W_resident = resident_weight_gb
W_batch(b) = swept_weight_gb

Audit requirements:

• resident parameter count source
• swept parameter count source
• direct byte-count source when exact byte fields are present
• explanation of which tensor prefixes or components are resident-only
• confirmation that the profile is for text decode bounds, not image/video prefill throughput

Weight Adapter: moe_distinct_experts

Use for MoE models where all experts are resident but each decode iteration touches an expected subset of routed experts.

{
  "kind": "moe_distinct_experts",
  "total_params_b": 26.5,
  "active_params_b": 4.0,
  "routed_experts": 128,
  "routed_experts_per_token": 8,
  "shared_experts_per_token": 0,
  "routing_model": "uniform_expected_distinct",
  "expert_param_b": null,
  "fixed_param_b": null,
  "shared_expert_notes": null
}

Required fields:

• kind
• total_params_b
• active_params_b
• routed_experts
• routed_experts_per_token
• routing_model

Optional fields:

• shared_experts_per_token
• expert_param_b
• fixed_param_b
• shared_expert_notes
• nonuniform_expert_notes

Convention:

• routed_experts is the number of routed experts, excluding shared experts.
• routed_experts_per_token is the number of routed experts selected per token.
• shared experts are always-on traffic and are folded into fixed_param_b unless a separate audited adapter variant is defined.
• active_params_b must include the parameters touched by one token, including shared experts when the architecture has them.

Default derived quantities:

E = routed_experts
k = routed_experts_per_token

expert_param_b =
  (total_params_b - active_params_b) / max(1, E - k)

fixed_param_b =
  max(0, active_params_b - k * expert_param_b)

m(b * rho) =
  E * (1 - (1 - k / E) ** (b * rho))

W_resident =
  total_params_b * weight_bytes_per_param

W_batch(b) =
  weight_bytes_per_param * (fixed_param_b + expert_param_b * m(b * rho))

This v1 adapter deliberately folds always-on shared expert traffic into fixed_param_b. If shared experts are not always-on, are nonuniform, or need a separate resident/traffic formula, the model needs a separate adapter variant instead of extra fields bolted onto moe_distinct_experts.

The adapter must preserve:

W_batch(b) <= W_resident

for every fitting batch. CI should enforce this invariant.

Audit requirements:

• total parameter count source
• active parameter count source
• routed expert count source
• routed experts per token source
• shared expert handling, if any
• whether uniform expected distinct expert routing is a documented assumption or a simplifying estimate
• proof that routed_experts excludes or includes shared experts; the profile must state the convention explicitly

If experts are materially nonuniform and no audited nonuniform formula exists, the model is unsupported for production bounds.

Weight Adapter: moe_distinct_experts_exact

Use for MoE packages where resident footprint and decode traffic are audited directly in decimal GB. This is preferred over moe_distinct_experts when stored dtypes differ by tensor group or when resident-only auxiliary modules must be separated from ordinary decode traffic.

{
  "kind": "moe_distinct_experts_exact",
  "resident_weight_gb": 688.57483936,
  "main_resident_weight_gb": 673.150611808,
  "auxiliary_resident_weight_gb": 15.424227552,
  "fixed_weight_gb": 19.082195296,
  "routed_expert_weight_gb": 2.554954752,
  "routed_experts": 256,
  "routed_experts_per_token": 8,
  "shared_experts_per_token": 1,
  "routing_model": "uniform_expected_distinct",
  "resident_parameter_scope": "safetensors_header_stored_fp8_bf16_f32",
  "traffic_scope": "ordinary decode excluding resident-only MTP layer"
}

Formula:

m(b * rho) = E * (1 - (1 - k / E) ^ (b * rho))
W_resident = resident_weight_gb
W_batch(b) = fixed_weight_gb + routed_expert_weight_gb * m(b * rho)

Audit requirements:

• exact resident byte source
• exact fixed traffic byte source
• exact per-routed-expert byte source
• proof that routed expert weights are uniform enough for expected-distinct aggregation
• explicit list of any resident-only auxiliary modules excluded from ordinary decode traffic

KV Adapter: full_context

Use for standard attention where every layer reads and stores full-context KV.

{
  "kind": "full_context",
  "layers": 32,
  "kv_heads": 8,
  "head_dim": 128
}

Required fields:

• kind
• layers
• kv_heads
• head_dim

Formula:

K_alloc(L_alloc) =
  2 * layers * kv_heads * head_dim * kv_store_bytes_per_scalar * L_alloc

K_read(L_read) =
  2 * layers * kv_heads * head_dim * kv_read_bytes_per_scalar * L_read

The engine converts bytes to decimal GB.

Audit requirements:

• layer count source
• KV head count source
• head dimension source
• confirmation that all counted layers use full-context attention

KV Adapter: layered_kv

Use for hybrid models, including local/global attention and sliding-window attention. This should be the default representation for mixed attention because it composes layer components explicitly.

{
  "kind": "layered_kv",
  "components": [
    {
      "kind": "full_context",
      "layers": 10,
      "kv_heads": 8,
      "head_dim": 256
    },
    {
      "kind": "sliding_window",
      "layers": 20,
      "kv_heads": 8,
      "head_dim": 256,
      "window_tokens": 4096
    }
  ]
}

Initial component kinds:

• full_context
• sliding_window
• fixed_state
• compressed_state

For a full_context component:

component_alloc(L_alloc) =
  2 * alloc_layers * kv_heads * head_dim * kv_store_bytes_per_scalar * L_alloc

component_read(L_read) =
  2 * read_layers * kv_heads * head_dim * kv_read_bytes_per_scalar * L_read

If alloc_layers or read_layers is omitted, it defaults to layers. Use explicit values for KV-sharing designs where fewer decoder layers store K/V than consume it during attention.

For a sliding_window component:

alloc_L = min(L_alloc, window_tokens)
read_L = min(L_read, window_tokens)

component_alloc(L_alloc) =
  2 * alloc_layers * kv_heads * head_dim * kv_store_bytes_per_scalar * alloc_L

component_read(L_read) =
  2 * read_layers * kv_heads * head_dim * kv_read_bytes_per_scalar * read_L

For a fixed_state component:

component_alloc(L_alloc) = alloc_gb_per_session
component_read(L_read) = read_gb_per_output_token

For a compressed_state component, the profile must provide an audited formula using the same shape as the standalone compressed_state adapter. A generic compression ratio is not enough for audited production bounds unless the source explicitly supports it.

Component outputs that use KV scalar counts are converted from bytes to decimal GB by the engine. Components that directly specify alloc_gb_per_session or read_gb_per_output_token, or formula fields named gb_per_1k_context_tokens, are already in decimal GB and must not be converted a second time.

Audit requirements:

• all component layer counts
• layer assignment rule, such as every Nth layer global
• KV heads and head dimension per component
• window size or compression formula
• whether allocation and read use the same effective context

KV Adapter: recurrent_state

Use for recurrent/state-space models where per-session state is fixed rather than full-context KV. Slowly growing recurrent state is a future adapter variant unless the growth formula is explicitly audited.

{
  "kind": "recurrent_state",
  "alloc_gb_per_session": 0.02,
  "read_gb_per_output_token": 0.002,
  "state_formula": "fixed"
}

Required fields:

• kind
• alloc_gb_per_session
• read_gb_per_output_token
• state_formula

Audit requirements:

• proof that the model does not require full-context KV
• source for state size
• source for read traffic approximation

If state traffic is not well understood, the model should remain unsupported.

KV Adapter: compressed_state

Use for latent, compressed, sparse, or otherwise non-full-context attention when there is audited evidence for the compression formula.

{
  "kind": "compressed_state",
  "alloc_formula": {
    "kind": "linear_context_ratio",
    "gb_per_1k_context_tokens": 0.01
  },
  "read_formula": {
    "kind": "linear_context_ratio",
    "gb_per_1k_context_tokens": 0.005
  }
}

Formula:

K_alloc(L_alloc) =
  alloc_formula.gb_per_1k_context_tokens * (L_alloc / 1000)

K_read(L_read) =
  read_formula.gb_per_1k_context_tokens * (L_read / 1000)

Both formulas return decimal GB. The read_formula ratio is context-depth traffic per output token; it scales with L_read, not with the number of output tokens generated by the session.

This adapter must be conservative. If a model claims million-token context via compressed attention but the actual read/allocation cost is unclear, do not invent a ratio. Mark the model unsupported until audited.

Embedded Serving Block

The serving block inside a model profile describes precision and runtime representation.

{
  "weight_format": "nvfp4",
  "weight_bytes_per_param": 0.5,
  "kv_store_format": "bf16",
  "kv_store_bytes_per_scalar": 2,
  "kv_read_format": "bf16",
  "kv_read_bytes_per_scalar": 2,
  "runtime_format": "format_agnostic_memory_bound",
  "dequantization_notes": "Memory-side bound charges stored weight bytes only unless an audited runtime overhead is added."
}

Required fields:

• weight_format
• weight_bytes_per_param
• kv_store_format
• kv_store_bytes_per_scalar
• kv_read_format
• kv_read_bytes_per_scalar

Initial weight_format values:

• bf16
• fp16
• fp32
• fp8
• fp4_fp8_mixed
• nvfp4
• mxfp4
• mixed_bf16_f32
• mixed_bf16_f16_f32
• int8
• int4
• q4
• q2_mixed
• gguf_quantized
• mlx_quantized
• unknown

Optional fields:

• runtime_format
• dequantization_notes
• notes

Rules:

• unknown weight or KV formats cannot be audited.
• runtime_format: format_agnostic_memory_bound means the profile claims only memory-side stored width and KV width, with no runtime-specific overhead.
• weight format and KV format must be separate.
• the serving block cannot change architecture fields.
• if runtime stores quantized weights but reads/dequantizes through a wider effective bandwidth path, that needs a separate audited runtime overhead field rather than an implicit change to architecture.
• evidence for serving fields belongs in the parent model profile evidence array.

Model Profile Status

• audited
• unsupported
• metadata_estimate

Production bounds require audited.

Status rules:

• audited means the self-contained model file has evidence and review for its embedded architecture and serving fields.
• unsupported means the model file exists but production bounds are disabled.
• metadata_estimate is diagnostic only and hidden from production comparisons by default.
• copied architecture or serving data must be recorded through base_model_proof or evidence records, but the copied values must still be embedded in the model file.
• A fine-tune can copy architecture data only if there is no evidence of structural changes.
• A merge can inherit only after review; default is unsupported.
• LoRA/adapters are unsupported unless the runtime merge/adapter traffic is explicitly modeled.
• Distilled, pruned, depth-changed, or architecture-modified repos need their own embedded architecture data.

Workload Settings Object

The engine receives workload settings from app defaults or UI state:

{
  "l_alloc_tokens": 100000,
  "l_read_tokens": 32000,
  "min_toks_per_session": 20,
  "overhead_gb": 8,
  "decode_policy": {
    "kind": "ordinary",
    "rho": 1
  }
}

Initial production decode setting:

• ordinary

Future or diagnostic decode settings:

• speculative
• custom_external

Required fields:

• l_alloc_tokens
• l_read_tokens
• min_toks_per_session
• overhead_gb
• decode_policy

Rules:

• ordinary must use rho = 1.
• Bounds Engine v1 must reject production settings with rho > 1.
• speculative is a future production setting. It may use rho > 1, but only after the schema defines draft model resident cost, draft weight traffic, draft KV/state traffic, verifier overhead, and acceptance-rate evidence.
• custom_external is diagnostic only and must not be used for production comparisons unless the UI labels the external assumption as diagnostic.
• raw rho must not be a standalone UI slider.

Bounds Result Profile

Bounds results should be serializable and testable.

{
  "engine_version": "1.0.0",
  "schema_version": "1.0.0",
  "status": "ok",
  "profile_resolution": {
    "repo": "RedHatAI/gemma-4-26B-A4B-it-NVFP4",
    "model_profile": "redhatai--gemma-4-26b-a4b-it-nvfp4",
    "model_profile_status": "audited"
  },
  "inputs": {
    "hardware_id": "nvidia-dgx-spark",
    "workload_settings": {
      "l_alloc_tokens": 100000,
      "l_read_tokens": 32000,
      "min_toks_per_session": 20,
      "overhead_gb": 8,
      "decode_policy": {
        "kind": "ordinary",
        "rho": 1
      }
    }
  },
  "trace": {
    "capacity_gb": 128,
    "bandwidth_gbps": 273,
    "overhead_gb": 8,
    "w_resident_gb": 12.0,
    "k_alloc_gb": 2.1,
    "free_gb": 108.0,
    "b_mem": 51,
    "single_session_toks_per_s": 120,
    "usable_batches_summary": {
      "count": 16,
      "min_batch": 1,
      "max_batch": 16
    },
    "b_star": 16,
    "w_batch_at_b_star_gb": 6.4,
    "k_read_gb": 0.42,
    "q_at_b_star_gb_per_output_token": 0.82,
    "aggregate_toks_per_s": 333,
    "per_session_toks_per_s": 20.8,
    "memory_power_ceiling_toks_per_s": 23700
  },
  "usable_batch": {
    "b_star": 16,
    "selection_rule": "max_aggregate_over_floor_qualified_batches"
  },
  "ceilings": {
    "single_session_toks_per_s": 120,
    "kv_aware_aggregate_toks_per_s": 333,
    "per_session_at_b_star_toks_per_s": 20.8,
    "memory_power_toks_per_s": 23700
  },
  "warnings": []
}

Required trace fields:

• capacity_gb
• bandwidth_gbps
• overhead_gb
• w_resident_gb
• k_alloc_gb
• free_gb
• b_mem
• single_session_toks_per_s
• usable_batches_summary
• b_star
• w_batch_at_b_star_gb
• k_read_gb
• q_at_b_star_gb_per_output_token
• aggregate_toks_per_s
• per_session_toks_per_s
• memory_power_ceiling_toks_per_s

The trace must be stable enough for snapshot tests and readable enough for the calculation details UI.

Profile Audit States

Use a strict state model:

draft
  -> audited
  -> deprecated

unsupported is a model profile state for repos that should fail closed.
metadata_estimate is diagnostic and never audited.

Definitions:

• draft: profile exists for discussion but cannot produce production bounds.
• audited: profile has evidence and review; can produce production bounds.
• deprecated: profile is retained for reproducibility but should not be used for new results.
• unsupported: repo has a model profile that intentionally disables production bounds.
• metadata_estimate: generated fallback, hidden from production comparison by default.

Bounds Engine Contract

Create docs/bounds-engine-v1.md before replacing the engine. That document must freeze the exact v1 math contract.

Units

• memory capacity: decimal GB
• memory bandwidth: decimal GB/s
• parameters: billions of parameters
• byte widths: bytes per parameter or bytes per KV scalar
• token counts: integer tokens
• rates: output tokens per second

Inputs

hardware:
  capacity_gb
  bandwidth_gbps

model profile:
  architecture.weight adapter
  architecture.KV/state adapter
  weight bytes
  KV store bytes
  KV read bytes

workload settings:
  L_alloc
  L_read
  r_star
  overhead_gb
  decode policy

Required Adapter Functions

W_resident(profile) -> GB
W_batch(batch, rho, profile) -> GB per decode iteration
K_alloc(L_alloc, profile) -> GB per session
K_read(L_read, profile) -> GB per output token
rho(settings) -> accepted tokens per session per iteration

Required Computation

free = C - W_resident - overhead

if free < 0:
  status = resident_not_fit

b_mem = floor(free / K_alloc(L_alloc))

if b_mem < 1:
  status = no_session_capacity

single_q = W_batch(1) / rho + K_read(L_read)
single_session = R / single_q

for b in 1..b_mem:
  q(b) = W_batch(b) / (b * rho) + K_read(L_read)
  aggregate(b) = R / q(b)
  per_session(b) = aggregate(b) / b

usable_batches = batches where per_session(b) >= r_star

if usable_batches is empty:
  status = no_floor

b_star = batch in usable_batches with max aggregate(b)

W_active =
  W_resident for dense weight adapters
  W_batch(1) for MoE and other batch-dependent weight adapters

memory_power_ceiling =
  rho * (C * R) / (K_alloc(L_alloc) * W_active) *
  (1 - (W_resident + overhead) / C)

Required Statuses

• unsupported_profile
• resident_not_fit
• no_session_capacity
• no_floor
• ok

The important semantic distinction is that no_floor is not a memory fit failure. It means sessions fit, but no fitting batch satisfies the minimum per-session rate.

Required Trace

Every successful or partially successful result must return a machine-readable root result with engine_version and schema_version, plus a stable trace object:

{
  "engine_version": "1.0.0",
  "schema_version": "1.0.0",
  "trace": {
    "capacity_gb": 128,
    "bandwidth_gbps": 273,
    "overhead_gb": 8,
    "free_gb": 108,
    "w_resident_gb": 12.0,
    "k_alloc_gb": 2.1,
    "b_mem": 51,
    "single_session_toks_per_s": 120,
    "usable_batches_summary": {
      "count": 16,
      "min_batch": 1,
      "max_batch": 16
    },
    "b_star": 16,
    "w_batch_at_b_star_gb": 6.4,
    "k_read_gb": 0.42,
    "q_at_b_star_gb_per_output_token": 0.82,
    "aggregate_toks_per_s": 333,
    "per_session_toks_per_s": 20.8,
    "memory_power_ceiling_toks_per_s": 23700
  }
}

The UI should render this trace in a collapsed calculation details section.

Data Files

Add these registry directories:

profiles/
  models/

The scraped catalog remains under assets/local-frontier-model-data.js, but it must not be the authority for serious bounds.

Schemas

Add JSON schemas under schemas/.

model-profile.schema.json

Required fields:

• id
• version
• schema_version
• repo
• status: draft, audited, deprecated, unsupported, or metadata_estimate
• model_family
• architecture
• serving
• evidence
• notes
• review, when status is audited
• unsupported_reason, when status is unsupported
• estimate_warning, when status is metadata_estimate

architecture.weight_adapter is a discriminated union by kind.

Initial variants:

• dense
• dense_resident_swept
• moe_distinct_experts
• moe_distinct_experts_exact

architecture.kv_adapter is a discriminated union by kind.

Initial variants:

• full_context
• layered_kv
• recurrent_state
• compressed_state

serving is an embedded object with required fields:

• weight_format
• weight_bytes_per_param
• kv_store_format
• kv_store_bytes_per_scalar
• kv_read_format
• kv_read_bytes_per_scalar

Optional fields:

• runtime_format
• dequantization_notes
• notes

bounds-result.schema.json

Required fields:

• engine_version
• schema_version
• status
• profile_resolution
• inputs
• trace
• ceilings
• usable_batch
• warnings

Required ceilings fields when status is ok:

• single_session_toks_per_s
• kv_aware_aggregate_toks_per_s
• per_session_at_b_star_toks_per_s
• memory_power_toks_per_s

Required usable_batch fields when status is ok:

• b_star
• selection_rule

Profile Loading

Loading order:

1. Find profiles/models/<model-id>.json for the selected repo.
2. Validate the model profile against model-profile.schema.json.
3. If status is audited, pass the self-contained profile to the engine.
4. If status is unsupported, return unsupported_profile with the embedded unsupported reason.
5. If status is metadata_estimate, allow it only in diagnostic mode.
6. If status is draft or deprecated, fail closed for production bounds.

The loader must never use repo-name heuristics for production bounds.

A model profile can copy architecture or serving data from a reviewed source only when the model profile records evidence:

• exact base-model relation
• compatible config
• artifact appears to change serving representation only
• no architecture-changing adapter, merge, pruning, distillation, or fine-tune claim that invalidates copied architecture data

Copied data is authoring-time reuse. The runtime still receives one self-contained model profile and does not resolve external profile references.

Audit Packet Generator

Add:

scripts/generate-profile-audit-packets.mjs

For each scraped model repo, generate an audit packet. The generator may also produce base-model group summaries for reviewer efficiency, but the output must preserve one candidate self-contained model profile per concrete repo:

• repo id
• base model chain
• card metadata
• config summary
• safetensors totals
• quantization metadata
• candidate embedded architecture block, if inferable
• candidate embedded serving block, if inferable
• missing evidence
• proposed model profile status
• proposed unsupported reason, when the repo should fail closed

Audit packets are review aids only. They must not automatically create audited model profiles.

Initial Model Profiles

Seed the registry from the worked examples in the source bounds doc:

1. gemma-4-26b-a4b

   • MoE weights
   • total around 26B
   • active around 4B
   • 128 routed experts
   • top-8 routing
   • hybrid local/global attention
   • serving examples include NVFP4 weights with BF16 KV

2. qwen3-6-35b-a3b

   • MoE weights
   • total around 35B
   • active around 3B
   • 256 routed experts
   • top-8 routing plus shared expert handling if supported by evidence
   • serving examples include NVFP4 weights

3. deepseek-v4-flash

   • MoE weights
   • total around 284B
   • active around 13B
   • 256 routed experts plus shared expert handling
   • compressed or sparse attention
   • Q2-style mixed serving block only where evidence supports it

After those, add dense families with simpler audited profiles:

• Llama dense families
• Qwen dense families
• Gemma dense families
• Phi dense families

Do not hand-write 654 unique architecture formulas. Do create one self-contained model profile or unsupported model profile for every scraped row. Coverage can be phased, but there must be no production fallback that treats an unreviewed row as supported.

Engine Implementation

Add TypeScript modules:

src/lib/bounds/
  engine.ts
  weight-adapters.ts
  kv-adapters.ts
  profiles.ts
  profile-loader.ts
  trace.ts

The engine must be pure:

• no DOM
• no network
• no repo-name logic
• deterministic for a given input

The static assets/bounds-engine.js should either be generated from the TS implementation or kept as a thin compatibility wrapper during migration.

UI Changes

Compare Page

Metric table layout is specified in docs/compare-table-metrics.md. That document is the source of truth for compare modes, default columns, and the advanced quantity toggle.

Main result should show:

• profile status
• resident fit
• memory-fit max batch, b_mem
• derived serving batch, b_star
• single-session decode ceiling
• KV-aware aggregate decode ceiling
• per-session rate at b_star
• Memory-power orientation ceiling

Remove raw Inspect concurrency from the main UI.

Optional future diagnostic:

• collapsed advanced forced-batch table
• clearly labeled as a policy what-if
• never presented as the primary result

Calculation Details

Add a collapsed section for supported models:

• hardware inputs: C, R, overhead
• workload inputs: L_alloc, L_read, r_star, decode policy
• model profile id and status
• W_resident
• K_alloc
• b_mem
• W_batch(b_star)
• K_read
• q(b_star)
• T = R / q
• per_session = T / b_star
• evidence links

For unsupported models, show:

Bounds unavailable: no audited adapter profile.

Model Pages

Add an Adapter Profile section:

• model profile status
• model profile id
• repo id
• weight adapter details
• KV/state adapter details
• precision details
• evidence links
• unsupported reason if unsupported

CI And Validation

Add scripts:

scripts/validate-profiles.mjs
scripts/test-profiled-bounds.mjs

CI gates:

• all profile files validate against schema
• every audited model profile has evidence
• every audited model profile has review
• every scraped model row has exactly one model profile file
• model profiles are self-contained and do not contain runtime architecture_profile or serving_profile references
• unsupported models cannot produce production bounds
• metadata estimates are disabled by default
• production workload settings with rho > 1 are rejected in Bounds Engine v1
• doc worked examples pass golden tests
• MoE W_batch(b) is nondecreasing with batch
• MoE W_batch(b) never exceeds W_resident
• dense W_batch(b) is constant with batch
• hybrid/sliding KV does not scale as full-context for local layers
• no index.html

Golden Tests

Use ranges rather than fake exact precision. The source doc gives the initial expected shape.

Example tests:

DGX Spark + NVIDIA Gemma 4 26B A4B NVFP4:
  single-session decode ceiling near 48 tok/s
  b_star near 12
  KV-aware aggregate near 246 tok/s
  note: the original bounds note used rounded 26B/4B NVFP4 bytes and produced
  the old 120 / 16 / 333 shape. The audited production target uses exact
  NVIDIA ModelOpt safetensors bytes, BF16 fixed language traffic, quantized
  routed expert traffic, and FP8 KV cache metadata from the served artifact.

Apple M5 Max 128GB + NVIDIA Gemma 4 26B A4B NVFP4:
  b_star near 97
  KV-aware aggregate near 1944 tok/s

DGX Spark + Qwen3.6 35B A3B NVFP4:
  single-session decode ceiling near 103 tok/s
  b_star near 13
  KV-aware aggregate near 275 tok/s
  note: the original bounds note used rounded 35B/3B NVFP4 bytes; the audited
  profile uses range-read safetensors-header bytes plus DeltaNet state.

DGX Spark + DeepSeek V4 Flash:
  single-session decode ceiling near 28 tok/s
  b_star near 3
  KV-aware aggregate near 62 tok/s
  note: the original bounds note used rounded 284B/13B Q2-style bytes; the
  audited profile uses exact q2-imatrix GGUF linked file size and tensor-index
  spans for the DS4 artifact.

Also test failure states:

• resident model too large
• no active session capacity
• sessions fit but no batch clears floor
• unsupported profile

Migration Slices

Slice 1: Documentation And Schemas

• add docs/bounds-engine-v1.md
• add model-profile.schema.json and bounds-result.schema.json
• add profile registry directory
• add empty or draft registry files
• add schema validation script

Slice 2: Pure Engine

• implement TS bounds engine
• implement dense and MoE weight adapters
• implement full-context and layered KV adapters
• add trace output
• add unit tests

Slice 3: Initial Audited Model Profiles

• add audited self-contained model profiles for reviewed Gemma 4 26B A4B repos
• add audited self-contained model profiles for reviewed Qwen3.6 35B A3B repos
• add audited self-contained model profiles for reviewed DeepSeek V4 Flash repos
• copy BF16, NVFP4, and Q-style serving blocks into those model profiles
• keep unreviewed rows unsupported

Slice 4: Loader And Fail-Closed UI

• implement model profile loader
• update compare page to use model profiles
• show unsupported state for missing or unsupported model profiles
• remove main concurrency control

Slice 5: Calculation Trace UI

• add calculation details on compare page
• add adapter profile sections on model pages
• include evidence links

Slice 6: Coverage Expansion

• generate audit packets for the 654 scraped rows
• audit highest-coverage base families first
• add one self-contained model profile per scraped row
• reuse copied architecture and serving blocks only through audit tooling and reviewed evidence
• keep unsupported model profiles unsupported with concrete unsupported reasons

Slice 7: Retire Old Metadata Bounds

• remove or quarantine generic metadata-derived serious bounds
• keep metadata estimates only in an explicitly labeled diagnostic mode
• make profile-backed bounds the only production path

Acceptance Criteria

The implementation is complete when:

• concurrency is derived, not controlled in the main UI
• every serious tok/s result has an audited self-contained model profile
• every scraped model row has an explicit model profile status or unsupported status
• every result has a calculation trace
• unsupported models fail closed
• schemas validate all profile data
• worked examples from the source doc pass golden tests
• no production bound relies on repo-name guessing
• deployed Space is verified on the final SHA