Security research PoC - gated access
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
This repository hosts a benign proof-of-concept model for a responsibly disclosed ModelScan scanner bypass. The payload only writes a marker file - no harmful action. Access is granted manually for vendor verification (Protect AI / huntr).
Log in or Sign Up to review the conditions and access this model content.
ModelScan Keras Scan Bypass — Lambda hidden in a nested sub-model
Responsible-disclosure security PoC for the Protect AI / huntr Model File Vulnerability program. The payload is benign (it only writes a marker file). Files are gated; access is granted for vendor verification.
TL;DR
case_nested_lambda_live.keras hides a Lambda layer inside a nested sub-model.
ModelScan only inspects top-level layers, so it never sees the Lambda and reports
clean. Keras deserializes the model recursively, reaches the nested Lambda, and runs
its (marshalled Python) code on load.
| Component | sees the nested Lambda? |
verdict |
|---|---|---|
| ModelScan (flat, top-level only) | no | No issues found |
| Keras (recursive deserialization) | yes | runs the Lambda code |
How it works
ModelScan's Keras scanner only flags a layer whose top-level class_name == "Lambda"
(modelscan/scanners/keras/scan.py; settings.py defines
unsafe_keras_operators = {"Lambda": "MEDIUM"}). The check does not recurse into
nested models.
By wrapping the Lambda inside a nested (e.g. Functional) sub-model, its class_name
at the top level is the sub-model, not Lambda — so ModelScan misses it. Keras, however,
walks the full config tree on load_model, instantiates the nested Lambda, and executes
the embedded Python.
Reproduce
pip install modelscan keras
# 1) Scanner says the file is clean:
modelscan -p case_nested_lambda_live.keras
# --- Summary ---
# No issues found
# 2) Keras runs the hidden Lambda on load (benign marker):
python - <<'EOF'
import keras
keras.config.enable_unsafe_deserialization()
keras.models.load_model("case_nested_lambda_live.keras")
EOF
enable_unsafe_deserialization() is the documented path for loading models that contain
Lambda / custom objects. The point is that the scanner declared the file safe.
Why it matters
Anyone using ModelScan as a gate (CI, model registries, pre-load scanning, MLOps) is told
a weaponized Keras model is clean. On load_model the nested Lambda runs arbitrary code,
i.e. remote code execution on the loading host.
Root cause and fix
The Keras / H5 / SavedModel detection is flat and non-recursive: it only flags
class_name == "Lambda" at the top level. Fix: walk the full nested config tree and flag
unsafe operators at any depth (and treat equivalent markers, e.g. TorchModuleWrapper
/ TFSMLayer, as unsafe as well).
Disclosure
Reported responsibly via the Protect AI / huntr Model File Vulnerability program against
protectai/modelscan. The PoC payload only writes a marker file; this repository is gated
and intended for vendor verification.
- Downloads last month
- 2