| # ModelSentry release validation |
|
|
| This directory contains the reproducible public release gate and its frozen 2026-08-11 evidence. |
|
|
| ## Contents |
|
|
| - `modelsentry_live_benchmark.py` scans the fixed 25-repository matrix through the public Gradio API and exercises negative inputs. |
| - `reports/2026-08-11-multi-repository-validation.md` records methodology, aggregate results, notable findings, UI verification, and limitations. |
| - `evidence/2026-08-11-live-benchmark.json` contains the structured per-target results used by the report. |
|
|
| The evidence contains public repository identifiers, aggregate timing, rule identifiers, package/version inventory, public |
| vulnerability identifiers, and scan coverage. It contains no Hugging Face token, password, authorization header, or |
| private repository content. |
|
|
| ## Run |
|
|
| From the repository root: |
|
|
| ```bash |
| python3 validation/modelsentry_live_benchmark.py \ |
| --output validation/evidence/modelsentry-live-benchmark.json |
| ``` |
|
|
| The runner targets the deployed public Space. A release passes when all repository scans complete, every exported SBOM |
| has CycloneDX 1.6 structure, negative inputs are rejected with sanitized messages, and the aggregate output matches the |
| expected scanner schema. |
|
|
| The 2026-08-11 baseline tested live Space commit `67fadc5a25d9887e3638e0d3a3851d9c4160ee85`. Future evidence files should |
| record the exact deployed commit, benchmark timestamp, test count, UI smoke result, and any matrix substitutions. |
|
|
