README.md

---
license: apache-2.0
tags:
  - security-research
  - proof-of-concept
  - modelscan
  - keras
library_name: keras
---

# Security Research PoC — ModelScan coverage gap for nested Keras `Lambda`

> **This repository contains a deliberate security-research proof-of-concept submitted
> through huntr (Protect AI / Palo Alto Networks) under the Model File Vulnerability
> program. It is NOT a usable model. Do not deploy it.**

## What this demonstrates

[ModelScan](https://github.com/protectai/modelscan) detects an unsafe top-level Keras
`Lambda` layer in a `.keras` archive, but **misses a structurally-equivalent unsafe
`Lambda` when it is nested inside a wrapper layer** (here `TimeDistributed`). The Keras
loader still reaches and deserializes that nested layer.

Two artifacts are included:

| File | Structure | ModelScan result |
| --- | --- | --- |
| `top-level-lambda.keras` | `Lambda` at top level (`config.layers[*]`) | **1 issue** (detected) |
| `nested-wrapper-lambda.keras` | same `Lambda` under `TimeDistributed.layer` | **0 issues** (missed) |
| `top-level-lambda.h5` | HDF5 control, top-level `Lambda` | **1 issue** (detected) |
| `nested-wrapper-lambda.h5` | HDF5, same `Lambda` under `TimeDistributed.layer` | **0 issues** (missed) |

The `.keras` and `.h5` variants exercise two separate ModelScan scanners
(`scanners/keras/scan.py` and `scanners/h5/scan.py`); each needs its own
recursive-walk fix.

Root cause: ModelScan's Keras scanner (`modelscan/scanners/keras/scan.py`,
`_get_keras_operator_names`) only iterates top-level `config.layers[*]` for
`class_name == "Lambda"` and does not recurse into wrapper `config.layer` (or
`Bidirectional.forward_layer` / `backward_layer`, etc.).

## This is NOT a Keras safe-mode bypass

Keras default `safe_mode=True` correctly **blocks both** models with a
`ValueError` about Lambda deserialization. The issue is purely a **scanner coverage
gap**: a tool meant to flag unsafe Keras artifacts returns a clean result for a model
that is structurally just as unsafe as one it flags. The risk is that a defender who
gates untrusted models on a clean ModelScan result is given false assurance and then
loads the artifact in an unsafe-deserialization context.

## The payload is benign

The nested `Lambda` writes a single marker file
`MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt` in the current working directory and returns
its input unchanged. It performs **no** network, shell, file-deletion, or otherwise
harmful action. The marker only proves the nested Lambda is reachable.

## Reproduce (only in an isolated environment you control)

```python
import keras
# default safe mode blocks both (expected):
keras.saving.load_model("nested-wrapper-lambda.keras")          # -> ValueError

# explicit unsafe load reaches the nested Lambda; calling the model runs it
# and writes the benign marker file:
m = keras.saving.load_model("nested-wrapper-lambda.keras", safe_mode=False)
import numpy as np
m(np.zeros((1, 2, 1), dtype="float32"))   # writes MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt
```

Scan both files with ModelScan to observe the differential (1 issue vs 0 issues).

## Suggested fix

ModelScan should recursively walk Keras configuration objects and flag `Lambda`
wherever it appears in nested layer-like fields (wrapper `layer`, bidirectional
`forward_layer`/`backward_layer`, nested preprocessing/pipeline layers, and other
`deserialize_keras_object()` targets), not only top-level `config.layers[*]`.

## Disclosure

Reported responsibly via huntr. Generated with Keras 3.15.0 (numpy backend).