granite-jev-1b-sampling / VALIDATION.md
jubba-io's picture
docs: align sampling artifact with audited selective replacement paper
57ff50e verified
|
Raw History Blame Contribute Delete
2.81 kB

Release validation

November evidence refresh

This documentation release aligns the model card and evidence guide with the audited SAR manuscript. Executables, client, dependency requirements, IBM weight/source pin and the greedy-plus-one-sample strict-0.05 policy are unchanged from the previous November bundle. The paper's new research controls use separate frozen runners; they are not extra models or policies in this download.

validation/maintenance-20261106.json records the new offline package/source checks and comparison with the previous anonymous artifact. validation/maintenance-20261102.json remains intact as historical evidence. No real-model inference, live Jev call or new GPU allocation belongs to this package-maintenance check. Optional inference tests are reported as skipped where their dependencies are absent, not treated as numerical validation.

The source publication receipt records the final combined-source checks and anonymous exact-revision download. The installation guide and publication metadata remain outside the hashed bundle to avoid circular revision references. Inference-library pins and their disclosed security boundaries are unchanged; see SECURITY.md.

Preserved real-model and provider evidence

validation/release-smoke.json and validation/release-e2e.json preserve the October 2026 Azure CPU integration checks. They are historical receipts, not new tests of this maintenance environment:

  • IBM Granite 4.0–1B FP32 greedy/sample token parity against the frozen R51 runtime passed on an authored arithmetic case (two tokens per mode), with unchanged weights and matched seed/payload mappings.
  • An authored full solve generated 43 native and 45 sampled tokens, called Jev 1.13.0, and returned the exact chosen native token sequence; the provider reported 638 input and 40 output tokens.
  • The original offline tests covered strict margin, reference isolation, token ownership, invalid judgments, credentials, cache isolation and failure records. Subsequent regressions added interruption, durable journaling, hooks and usage accounting. See the existing source tests for the full contract.

The previous validation record preserves the detailed history. Neither those small integration cases nor the current unit tests establish answer quality, current provider availability, GPU/cross-device equality or deployment throughput. Scientific evidence is separate and summarized in RESULTS.md.