README.md

# ModelScan Bypass — Arbitrary Code Execution in `.keras` via Nested Lambda Layer

Proof-of-concept for a **Huntr Model File Format** report.

`model.keras` runs code on `keras.saving.load_model(..., safe_mode=False)` while
**ModelScan reports it clean**. The malicious Lambda layer is hidden **one level deep inside a
nested sub-model**. ModelScan's `KerasLambdaDetectScan` only inspects the flat, top-level
`config.layers` list (and matches `class_name == "Lambda"` exactly), so it never sees the
nested Lambda — but Keras deserializes sub-models recursively and executes it on load.

## Files
- `model.keras` — the PoC model file (benign payload: writes `keras_poc_executed.txt`)
- `exploit.py` — builds a flat model (detected) and the nested model (missed), scans + loads both
- `README.md` — this file

## Reproduce
Use **Python 3.10+** (verified on 3.12 / TensorFlow 2.21 / Keras 3.14 / modelscan 0.8.8).
TensorFlow does not run on the EOL Python 3.9 — on macOS arm64 it aborts at import with
`mutex lock failed`, which is unrelated to this issue.

```bash
# 0) Get Python 3.10+ if needed:
#    macOS:        brew install python@3.12
#    Debian/Ubuntu: sudo apt-get install -y python3.12 python3.12-venv
python3.12 --version                     # -> Python 3.12.x

# 1) Clean virtual environment so `python` is 3.12, not the system 3.9
python3.12 -m venv venv
source venv/bin/activate                 # Windows: venv\Scripts\activate
python --version                         # -> Python 3.12.x
pip install 'modelscan[tensorflow]' tensorflow keras

# 2) Scanner says it is safe:
modelscan -p model.keras                 # ->  No issues found
# 3) Loading it executes code:
python -c "import keras; keras.saving.load_model('model.keras', safe_mode=False, compile=False)"
ls keras_poc_executed.txt                # marker proves code ran on load
# or run the full demo (flat=detected, nested=bypass):
python exploit.py                        # -> No issues found (BYPASS) + CODE EXECUTED
```

## Why it works
`modelscan/scanners/keras/scan.py` (`_get_keras_operator_names`):
```python
lambda_layers = [
    layer.get("config", {}).get("function", {})
    for layer in model_config_data.get("config", {}).get("layers", {})   # TOP LEVEL ONLY
    if layer.get("class_name", {}) == "Lambda"                           # exact match
]
```
It iterates only the outer `config.layers` and never recurses. A Lambda inside a sub-model
(top-level `class_name` is `Functional`, not `Lambda`) is invisible to the scan.

## Why this is a scanner bug (not `safe_mode` misuse)
The report is not "Keras runs Lambda layers" (known). It is that ModelScan's `KerasLambdaDetectScan`
flags a flat Lambda (1 issue) but returns 0 issues for an identical Lambda nested one level deep.
Both files are equally dangerous; the scanner certifies one as clean. That false negative is the
bug, independent of how the file is later loaded.

## Impact
ModelScan is used to gate untrusted models in MLOps pipelines / model hubs. This file passes
the scan as clean yet achieves arbitrary code execution on load — defeating the control.
Verified on modelscan 0.8.8 / Keras 3.14.1 / TensorFlow 2.21.

The payload here is benign (writes a marker file). Swap the command inside the Lambda in
`exploit.py` to confirm real command execution.