| ---
|
| license: apache-2.0
|
| library_name: kernels
|
| tags:
|
| - kernel
|
| - inference
|
| - energy
|
| - nvml
|
| - governance
|
| - provenance
|
| - audit
|
| - receipts
|
| - tokens-per-joule
|
| - sovereign-ai
|
| ---
|
|
|
| > **SZL Holdings** · Doctrine v11 · Λ = Conjecture 1 (advisory, never "green"/theorem) · canonical [a-11-oy.com](https://a-11-oy.com)
|
|
|
| # governed-inference-meter
|
|
|
| > # ⚠️ DEPRECATED — consolidated into [`szl-energy-attest`](https://github.com/szl-holdings/szl-energy-attest)
|
| >
|
| > **This repository is deprecated.** Its live inference meter and meter-specific
|
| > attestation, hardened receipt-chain, PCGI spine, and signing-facing compatibility
|
| > APIs were folded into the canonical
|
| > [`szl_energy_attest.inference_meter`](https://github.com/szl-holdings/szl-energy-attest/tree/4d8d105c3d5ea67b5eb25826e8a2a35ca35f4043/szl_energy_attest/inference_meter)
|
| > package in verified merge
|
| > [`4d8d105c3d5ea67b5eb25826e8a2a35ca35f4043`](https://github.com/szl-holdings/szl-energy-attest/commit/4d8d105c3d5ea67b5eb25826e8a2a35ca35f4043).
|
| >
|
| > Three legacy modules (`_energy.py`, `_policy.py`, `_receipt.py`) are exact
|
| > hash-preserving copies; `_attest.py` and `_spine.py` carry only bounded
|
| > package/import and install-guidance rewrites. The immutable
|
| > [`MIGRATION_PROVENANCE.json`](https://github.com/szl-holdings/szl-energy-attest/blob/4d8d105c3d5ea67b5eb25826e8a2a35ca35f4043/MIGRATION_PROVENANCE.json)
|
| > records every source and destination digest, and the
|
| > [migration regressions](https://github.com/szl-holdings/szl-energy-attest/blob/4d8d105c3d5ea67b5eb25826e8a2a35ca35f4043/tests/test_inference_meter_migration.py)
|
| > verify that boundary.
|
| >
|
| > **Boundary:** the successor's root energy receipt remains a different schema;
|
| > compatibility does not imply schema equality, a configured signing key, or a
|
| > signed receipt. This legacy repository and its Hugging Face artifact remain
|
| > readable for provenance and rollback, but are not the target for new
|
| > integrations. Archiving requires a separate owner decision after pointer and
|
| > inbound-link gates are evidenced. See [`DEPRECATED.md`](./DEPRECATED.md).
|
| > Λ remains **Conjecture 1 (advisory, uniqueness OPEN)** — never upgraded to proven.
|
|
|
| **Energy-metered, governed inference receipts.** A lightweight, dependency-light
|
| Python utility (and Hugging Face *universal* kernel) that wraps any inference
|
| call and emits a **governed, energy-metered, tamper-evident receipt**:
|
|
|
| - **measures GPU energy** via NVIDIA NVML (power/energy readback) integrated
|
| over wall-time → **joules**,
|
| - computes **tokens-per-joule**,
|
| - runs a pluggable, **advisory policy gate** (allow/deny; defaults to allow),
|
| - and emits a **SHA-256 hash-chained JSON receipt** so a sequence of calls is
|
| independently auditable.
|
|
|
| It is the energy + governance counterpart to
|
| [`SZLHOLDINGS/szl-governed-norm`](https://huggingface.co/SZLHOLDINGS/szl-governed-norm)
|
| — provenance at the inference boundary, in the spirit of the
|
| [a11oy](https://a-11-oy.com) governed-AI platform: **receipts, not capability
|
| claims.**
|
|
|
| > **Why this exists.** Browse the [Kernel Hub](https://huggingface.co/models?other=kernel)
|
| > and you find performance kernels — attention, activations, GEMM, norms. There
|
| > is **no energy-metering + governance kernel**. Teams running inference in
|
| > sovereign, regulated, or cost/carbon-sensitive contexts measure tokens/joule
|
| > and keep audit trails *by hand*. This utility does both in one wrapped call,
|
| > and degrades honestly when no GPU energy readback is available.
|
|
|
| ---
|
|
|
| ## Honest scope (read this first)
|
|
|
| This project follows a strict honesty doctrine. **Λ (the governance trust
|
| quantity) is Conjecture 1 — advisory, not a theorem. Trust is never 100%.**
|
|
|
| - **MEASURED only with NVML.** Energy is real **only** when NVML is present and
|
| grants power/energy readback. Without it the receipt is labeled
|
| `mode="unmeasured"` and `joules` / `tokens_per_joule` are `null`. **We never
|
| fabricate a joule figure.**
|
| - **Board-level power.** NVML reports whole-board power (compute die + memory +
|
| losses). We report what the hardware reports and say so. No modeling, no
|
| scaling factors.
|
| - **The policy gate is advisory and host-enforced.** It records an allow/deny
|
| decision into the receipt. It does **not**, and cannot, enforce anything by
|
| itself — your host must actually skip a denied call. The bundled `meter()`
|
| wrapper *does* fail-safe (it will not execute a denied call), but downstream
|
| enforcement is still your responsibility.
|
| - **The receipt digest is an integrity fingerprint, not a signature.** It is a
|
| SHA-256 over the canonical record body and makes tampering *evident*. It does
|
| **not** prove authorship. Cryptographic signing (e.g. DSSE/Sigstore) is a
|
| separate, out-of-band concern, intentionally not done here.
|
| - **This is a metering + receipt utility, not a safety guarantee.**
|
|
|
| ---
|
|
|
| ## Install / load
|
|
|
| **From the Hugging Face Hub** (universal kernel — runs on CPU and CUDA):
|
|
|
| ```python
|
| from kernels import get_kernel
|
| gim = get_kernel("SZLHOLDINGS/governed-inference-meter")
|
| ```
|
|
|
| **From PyPI-style source** (zero hard dependencies; add `pynvml` for real
|
| energy):
|
|
|
| ```bash
|
| pip install kernels # to load via get_kernel
|
| # real GPU energy measurement additionally needs NVML bindings:
|
| pip install pynvml
|
| ```
|
|
|
| ---
|
|
|
| ## Usage
|
|
|
| ```python
|
| from kernels import get_kernel
|
| gim = get_kernel("SZLHOLDINGS/governed-inference-meter")
|
|
|
| print(gim.__version__)
|
| print(gim.capability_report()) # what energy measurement is possible here
|
|
|
| # Wrap ANY inference callable. You tell the meter the token counts.
|
| def run(prompt):
|
| # ... your real model.generate(...) call here ...
|
| return "the model's response text"
|
|
|
| receipt, output = gim.meter(
|
| run, args=("hello",),
|
| model="my-llm-7b",
|
| tokens_in=2, tokens_out=7,
|
| )
|
|
|
| print(receipt["mode"]) # 'measured-energy' | 'measured-power-integral' | 'unmeasured'
|
| print(receipt["joules"]) # float, or None when unmeasured
|
| print(receipt["tokens_per_joule"]) # float, or None when unmeasured
|
| print(receipt["policy_decision"]) # 'allow' | 'deny'
|
| print(receipt["digest"]) # SHA-256 over the canonical record body
|
| print(gim.receipt_verify()) # (ok, depth, first_break_seq) over the chain
|
| ```
|
|
|
| ### A custom policy gate (advisory)
|
|
|
| ```python
|
| def my_gate(ctx):
|
| # ctx has model, tokens_in, tokens_out, args, kwargs, ts
|
| if ctx["tokens_in"] > 8192:
|
| return ("deny", "prompt exceeds governed token budget")
|
| return ("allow", "within budget")
|
|
|
| receipt, output = gim.meter(run, args=("hi",), model="m",
|
| tokens_in=2, tokens_out=7, policy=my_gate)
|
| ```
|
|
|
| A gate may return a `PolicyResult`, a `(decision, reason)` tuple, a bool, or a
|
| string. **It runs fail-closed**: if your gate raises, the call is denied with
|
| the exception text as the reason — a buggy policy can never silently allow.
|
|
|
| ### Per-request chain (no global-state contention)
|
|
|
| ```python
|
| chain = gim.ReceiptChain()
|
| gim.meter(run, args=("a",), model="m", tokens_in=1, tokens_out=4, chain=chain)
|
| gim.meter(run, args=("b",), model="m", tokens_in=1, tokens_out=6, chain=chain)
|
| print(chain.verify()) # tamper-evident over YOUR chain only
|
| print(chain.to_jsonl()) # export the chain for offline audit
|
| ```
|
|
|
| ---
|
|
|
| ## MEASURED vs. `unmeasured` — what you get
|
|
|
| | Environment | `mode` | `joules` | `tokens_per_joule` |
|
| |---|---|---|---|
|
| | NVIDIA GPU with energy counter (`nvmlDeviceGetTotalEnergyConsumption`) | `measured-energy` | hardware accumulator delta | computed |
|
| | NVIDIA GPU, power readback only (`nvmlDeviceGetPowerUsage`) | `measured-power-integral` | trapezoidal integral of power samples | computed |
|
| | No GPU / no driver / no permission / no `pynvml` | `unmeasured` | `null` | `null` |
|
|
|
| ### Sample receipt — `unmeasured` (illustrative; this build env has no GPU)
|
|
|
| > **SAMPLE / illustrative.** Produced on a CPU-only box. Because NVML is
|
| > unavailable, energy is honestly `unmeasured` and `joules` is `null` — exactly
|
| > the honest-degrade behavior. **No energy number is invented.**
|
|
|
| ```json
|
| {
|
| "seq": 0,
|
| "model": "my-llm-7b",
|
| "tokens_in": 2,
|
| "tokens_out": 7,
|
| "mode": "unmeasured",
|
| "joules": null,
|
| "wall_seconds": 0.004182,
|
| "tokens_per_joule": null,
|
| "policy_decision": "allow",
|
| "policy_reason": "default allow_all gate (no policy configured)",
|
| "prev": "0000000000000000000000000000000000000000000000000000000000000000",
|
| "digest": "<sha256 of the canonical body>",
|
| "ts": 1750000000.0
|
| }
|
| ```
|
|
|
| On a real NVIDIA GPU the same call would carry e.g.
|
| `"mode": "measured-energy"`, a positive `"joules"`, and a computed
|
| `"tokens_per_joule"`. We do **not** print example GPU numbers here because this
|
| build environment cannot measure them, and inventing them would violate the
|
| honesty doctrine. Run `gim.selfcheck()` on your own hardware to see your numbers.
|
|
|
| ---
|
|
|
| ## Self-test
|
|
|
| ```python
|
| import governed_inference_meter as gim
|
| print(gim.selfcheck()) # functional check (NOT a benchmark); no fabricated energy
|
| ```
|
|
|
| `selfcheck()` runs a metered allow call, a denied call (verifying it does not
|
| execute), checks tokens/joule honesty, verifies the hash chain, and confirms
|
| that mutating a past record is detected. It requires no GPU.
|
|
|
| ---
|
|
|
| ## Attestation & compliance evidence (interop layer)
|
|
|
| A receipt is only as useful as the tools that can *carry* it. This module renders
|
| any receipt into the formats the wider ecosystem already understands — without
|
| changing a single measured value.
|
|
|
| ```python
|
| import governed_inference_meter as gim
|
|
|
| rec, out = gim.meter(run, args=("hi",), model="my-llm", tokens_in=2, tokens_out=7)
|
|
|
| # 1) The receipt as an in-toto Statement v1 — the exact payload that
|
| # Sigstore / DSSE / SCITT tooling signs and stores in a transparency log.
|
| stmt = gim.to_intoto_statement(rec) # SLSA-shaped predicate, our own type URI
|
|
|
| # 2) EU AI Act / NIST AI RMF controls this receipt provides EVIDENCE for,
|
| # with an explicit does_not_establish note per control (honest, not a cert).
|
| ev = gim.compliance_evidence(rec)
|
|
|
| # 3) Confirm the Statement is cryptographically bound to this exact receipt.
|
| ok, why = gim.verify_statement(stmt, rec) # -> (True, "ok")
|
| ```
|
|
|
| Honest boundaries (doctrine):
|
|
|
| - The predicate uses **our own** `predicateType` URI and is only SLSA-*shaped*
|
| for auditor recognizability — it is **not** a claim of official SLSA
|
| conformance. Signing (DSSE/Sigstore) is out-of-band; this emits the unsigned
|
| Statement payload a signer would then cover.
|
| - Energy fields are copied **verbatim**. On an unmeasured receipt, energy-dependent
|
| controls (e.g. `NIST-AI-RMF-MEASURE-2.x`) report **`UNAVAILABLE`** — never a
|
| fabricated joule. Logging / record-keeping controls (EU AI Act Art. 12 & 19)
|
| are supported regardless of GPU.
|
| - A receipt is **evidence** toward a control, never a conformity assessment,
|
| certification, or safety guarantee.
|
|
|
| ---
|
|
|
| ## Canonical PCGI receipt (spine fold)
|
|
|
| The meter is also a first-class **Proof-Carrying Governed Intelligence (PCGI)**
|
| receipt producer on the org-canonical [`szl-receipt`](https://huggingface.co/SZLHOLDINGS)
|
| spine. One call folds a metered inference into a single signed receipt that binds
|
| **model id + input digest + output digest + governing policy id + energy** — the
|
| same shape every other decision producer emits, so provenance unifies.
|
|
|
| ```python
|
| import governed_inference_meter as gim
|
| from szl_receipt import generate_keypair
|
|
|
| priv, pub = generate_keypair() # or sign_key=None for UNSIGNED-honest
|
|
|
| # End-to-end: meter the call AND emit ONE canonical szl-receipt for it.
|
| env, out = gim.meter_szl_receipt(
|
| run, args=("hi",), model="my-llm",
|
| policy_id="default-allow", sign_key=priv, organ="meter",
|
| )
|
| ok, why = gim.verify_szl_receipt(env, pub) # -> (True, "ok")
|
| stmt = gim.to_statement(env) # in-toto Statement v1, SLSA-shaped
|
| ok2, _ = gim.verify_szl_statement(stmt, env) # bound to this exact receipt
|
|
|
| # Or fold an existing meter receipt you already have:
|
| rec, out = gim.meter(run, args=("hi",), model="my-llm", tokens_in=2, tokens_out=7)
|
| env = gim.from_meter_receipt(rec, input="hi", output=out, policy_id="default-allow")
|
| ```
|
|
|
| Honest boundaries (doctrine):
|
|
|
| - Reuses szl-receipt's **canonicalization + signing + in-toto shapes** — it does
|
| **not** invent a new receipt shape.
|
| - **Energy** is bound **verbatim** only when the meter actually measured it
|
| (NVML present). Otherwise `energy.joules` is the literal string
|
| **`"UNAVAILABLE"`** and `energy.measured` is `False` — the meter is the one
|
| place in the spine where energy *can* be real, and it is never fabricated.
|
| - The canonical body is **deterministic**: identical inputs serialize to
|
| byte-identical canonical JSON (no timestamps in the body).
|
| - Keyless => **UNSIGNED-honest** (`signed=False`); a signature is never faked.
|
| - The receipt is **evidence** binding a decision, **not** a proof the model's
|
| output is correct.
|
|
|
| Requires the shared `szl-receipt` library (install extra `[sign]`); the import is
|
| lazy, so importing this package stays zero-hard-dependency.
|
|
|
| ---
|
|
|
| ## What's in the repo
|
|
|
| ```
|
| build.toml # Kernel Hub universal-kernel manifest
|
| build/torch-universal/governed_inference_meter/
|
| __init__.py # meter() / metered() wrappers, selfcheck(), accessors
|
| _energy.py # NVML energy + power-integral measurement, honest degrade
|
| _receipt.py # SHA-256 hash-chained, tamper-evident receipts
|
| _policy.py # advisory policy gate (allow_all default, fail-closed)
|
| _attest.py # in-toto/SLSA-shaped Statements + EU AI Act / NIST AI RMF evidence
|
| _spine.py # PCGI spine fold: metered inference -> ONE canonical szl-receipt
|
| metadata.json
|
| pyproject.toml # also pip-installable from source
|
| tests/test_meter.py # runs on CPU, no GPU needed
|
| tests/test_attest.py # attestation + compliance, no GPU needed
|
| tests/test_spine.py # canonical PCGI receipt fold, no GPU needed
|
| LICENSE # Apache-2.0
|
| ```
|
|
|
| ---
|
|
|
| ## Doctrine & honesty disclaimer
|
|
|
| > SZL Holdings · governed, energy-metered inference receipts · **MEASURED only
|
| > with NVML** · the policy gate is **advisory (host-enforced)** · **Λ =
|
| > Conjecture 1** (advisory, *not* a theorem) · **trust never 100%** · honesty
|
| > over checklist. This is a metering + receipt utility, **not** a safety
|
| > guarantee. No fabricated benchmarks; energy is reported only when physically
|
| > measured.
|
|
|
| **License:** Apache-2.0 · **Maintainer:** Stephen Lutar
|
| <stephenlutar2@gmail.com> · **Platform:** [a-11-oy.com](https://a-11-oy.com) — the
|
| governed-inference substrate for hard missions.
|
|
|
