validation/README.md

ModelSentry release validation

This directory contains the reproducible public release gate and its frozen 2026-08-11 evidence.

Contents

• modelsentry_live_benchmark.py scans the fixed 25-repository matrix through the public Gradio API and exercises negative inputs.
• reports/2026-08-11-multi-repository-validation.md records methodology, aggregate results, notable findings, UI verification, and limitations.
• evidence/2026-08-11-live-benchmark.json contains the structured per-target results used by the report.

The evidence contains public repository identifiers, aggregate timing, rule identifiers, package/version inventory, public vulnerability identifiers, and scan coverage. It contains no Hugging Face token, password, authorization header, or private repository content.

Run

From the repository root:

python3 validation/modelsentry_live_benchmark.py \
  --output validation/evidence/modelsentry-live-benchmark.json

The runner targets the deployed public Space. A release passes when all repository scans complete, every exported SBOM has CycloneDX 1.6 structure, negative inputs are rejected with sanitized messages, and the aggregate output matches the expected scanner schema.

The 2026-08-11 baseline tested live Space commit 67fadc5a25d9887e3638e0d3a3851d9c4160ee85. Future evidence files should record the exact deployed commit, benchmark timestamp, test count, UI smoke result, and any matrix substitutions.