README.md

---
title: ModelSentry
emoji: 🛡️
colorFrom: indigo
colorTo: red
sdk: gradio
sdk_version: 6.22.0
python_version: "3.12"
app_file: app.py
pinned: false
license: apache-2.0
short_description: Supply-chain and provenance auditor for HF repositories
tags:
  - security
  - supply-chain
  - model-audit
  - static-analysis
  - cpu
---

# ModelSentry

ModelSentry performs bounded, evidence-first static inspection of public Hugging Face Models and Spaces. It resolves an
immutable commit, downloads only allowlisted text/configuration files under strict limits, and reports licensing,
dependency, code-execution, privacy, telemetry, secret-handling, container, documentation, provenance, artifact, and
reproducibility concerns.

**Public scanner:** https://huggingface.co/spaces/SeedOfEvil/ModelSentry-HF-Auditor

Enter `model:owner/repository`, `space:owner/repository`, or a public Hugging Face URL. ModelSentry produces an
interactive evidence report, downloadable `modelsentry.scan.v4` JSON, and a CycloneDX 1.6 SBOM.

Exactly pinned PyPI packages are correlated against OSV using a bounded batch query. Floating constraints, Git sources,
and direct URLs are shown as unresolved inventory and are never queried as though they were exact versions. Every scan
also produces a downloadable CycloneDX 1.6 SBOM.

## Validated release

The frozen v1 validation matrix covered 10 Models and 15 Spaces, including every public SeedOfEvil repository available
on the benchmark date:

- 25/25 scans completed successfully.
- 25/25 CycloneDX 1.6 SBOMs passed structural validation.
- Median wall-clock latency was 0.749 seconds; maximum latency was 5.057 seconds on CPU Basic.
- 97 declared packages, 51 deduplicated vulnerability records, and 77 evidence-linked finding groups were inventoried.
- Invalid external URLs and nonexistent repositories were rejected.
- A dense `microsoft/TRELLIS` browser report passed the single-scroll-owner UI smoke.

The reproducible runner, full report, and raw structured evidence live under [`validation/`](validation/README.md).

## Security boundary

- Repository code is never imported or executed.
- Model weights and binary artifacts are not downloaded.
- Only public Hugging Face repositories are accepted.
- Downloads are restricted by file type, file count, and byte size.
- Credential-like evidence values are redacted before display.
- Scanned contents are not intentionally logged or durably retained.
- Reports describe evidence and uncertainty; they do not certify a repository as safe.

## Report coverage

Reports distinguish **Passed**, **Attention**, **Partial**, **Not applicable**, and **Not checked** instead of treating zero detected
findings as proof of safety. Model reports inventory published formats and sizes, validate safetensors shard references,
identify architecture metadata, inspect model-card completeness and usage examples, normalize modified base-model tags,
and follow public upstream model metadata through two provenance levels for license review.

## Precision-first analysis

- Python behavior rules use the AST, so comments and quoted examples are not treated as executable code.
- Remote-code loading distinguishes immutable revision pins from floating code.
- Subprocess findings distinguish shell execution from fixed non-shell calls with timeouts.
- Environment writes, ordinary reads, secret-like reads, and dotenv loading are separate behaviors.
- Dependency findings distinguish floating Git refs, versioned release assets, and hash-verified artifacts.
- Repeated evidence is grouped by rule with complete locations retained in the JSON report.
- File-cap scans are explicitly marked **Partial** and prioritize manifests, entry points, and security-sensitive source.
- OSV correlation is capped at 50 exact packages and 40 vulnerability records, cached for 15 minutes, and marked Partial
  or Not checked instead of silently treating API failures or limits as clean results.
- CycloneDX components retain exact, floating, direct, and VCS declarations with their manifest evidence.

Repositories can add a `.modelsentry.json` control declaration for expected outbound domains:

```json
{
  "expected_outbound_domains": ["huggingface.co", "api.osv.dev"]
}
```

This changes matching network findings to **Documented**, not **Passed**. ModelSentry still reports the capability and
does not infer that transmitted content or retention practices are safe.

## Current scope

ModelSentry supports public Models and Gradio/Docker Spaces and emits `modelsentry.scan.v4` JSON plus CycloneDX 1.6.
Dataset scanning, JavaScript dependency inspection, deeper license compatibility policy, and binary tensor inspection
remain future work.

## Interpreting a report

- **Complete** means every eligible item within the documented scanner boundary was inspected.
- **Partial** means a file, byte, package, vulnerability, or upstream traversal cap was reached; the report states why.
- **Not applicable** means no supported evidence existed for that check, not that the repository is clean.
- **Not checked** means ModelSentry made no conclusion for that category.
- Critical, High, Medium, Low, and Info findings are prioritized review evidence. They are not an exploitability verdict
  or a certification that a repository is safe.

## Release policy

Changes must pass the local regression suite. Scanner releases additionally run the frozen public matrix, validate every
downloaded SBOM, exercise negative inputs, and complete a browser smoke against a dense result. Low-visibility controls
may be refreshed when repositories disappear, but the portfolio and high-signal controls remain stable for comparison.

## Local development

```bash
python3 -m unittest discover -s tests -v
python3 app.py
```

Run the public release matrix after deployment:

```bash
python3 validation/modelsentry_live_benchmark.py \
  --output validation/evidence/modelsentry-live-benchmark.json
```

See [`CHANGELOG.md`](CHANGELOG.md) for release history.

## License

The ModelSentry source code is licensed under Apache-2.0. Reports may quote short, redacted evidence snippets from the
repository being inspected; the scanned repository remains governed by its own license.