Fix MoE expert traffic, surface search limits, retire dennard naming

#2
by osolmaz HF Staff - opened

Summary

Follow-up to the correctness audit of the bounds engine, and terminology housekeeping after the theory note retired the Dennard eponym in favor of "memory power".

Engine correctness

  • Legacy MoE adapter fix. The derived per-expert size divided the non-active parameters by E - k - s, but shared experts are always-on and never part of the non-active pool. The denominator is now E - k, with shared experts folded into the fixed term. Before the fix, W_batch saturated above the total resident weight for shared-expert MoE models, violating the upper-bound contract on 61 scraped catalog rows. A synthetic regression test pins W_batch(1) = active and W_batch(inf) = total exactly.
  • Zero-allocation edge case. A profile with zero per-session KV/state allocation now reports session capacity at the search cap with a warning, instead of the incorrect no_session_capacity.
  • Silent batch-search truncation. When the batch search stops at the 10000 cap with the per-session floor still satisfied, the result now carries a warning that the reported KV-aware ceiling may understate the true bound.

Data correction caught by the new validator check

validate-profiles now checks that saturated expert traffic (fixed + E * expert) stays within the swept resident footprint for both exact and derived MoE adapters. This immediately caught nvidia--glm-5-2-nvfp4: its routed expert group was recorded as 1.668024072 GB from a tensor grouping that double-counted shared-expert tensors inside the routed sum. Re-derived from the same pinned shard headers (all 47 shards, commit aec724e8): 256 uniform groups of exactly 1.592526600 GB, so fixed + 256 experts now reproduces the 442.986259968 GB main resident total exactly. The catalog row, profile evidence, and golden tests were updated; single-session throughput for this model rises slightly (e.g. DGX Spark 2.092 -> 2.101 tok/s) because modeled traffic decreases.

Housekeeping: engine 1.1.0 field renames

All "Dennard" naming is replaced by "memory power" across the engine contract, UI, docs, and tests, matching the updated theory note. Computed values are identical.

  • trace.dennard_ceiling_toks_per_s -> trace.memory_power_ceiling_toks_per_s
  • ceilings.dennard_toks_per_s -> ceilings.memory_power_toks_per_s
  • top-level dennard -> memoryPowerCeiling
  • UI column "Dennard bound" -> "Memory-power bound"

docs/profile-audit-progress.md is left untouched as a historical log.

Test plan

  • npm run check (format, lint, typecheck, vitest, legacy engine tests, profile + model-data validation, build) passes.
  • New regression tests: legacy MoE saturation, zero-alloc warning path, batch-search truncation warning path.
  • All 638 profiles validate under the new saturation check.
osolmaz changed pull request status to open
osolmaz changed pull request status to merged

Sign up or log in to comment