pragnyanramtha/keras-native-safe-mode-public-function-modelscan-bypass

Benign Keras Native Safe-Mode Public Function PoC

This repository contains a benign security proof of concept for Keras Native .keras model artifacts. It demonstrates a nested keras.layers.Lambda whose function config points to the public Keras API keras.visualization.plot_image_gallery.

Keras 3.14.1 loads the artifact with safe_mode=True. When inference is run, the model writes a benign marker PNG named safe_mode_public_function_marker.png in the working directory. Protect AI ModelScan 0.8.8 reports zero issues for the nested artifact, while the included top-level control model with the same Lambda is detected.

This PoC is non-destructive: it only writes a small local PNG marker.

Files

• safe_mode_public_function_nested.keras: scanner-clean nested PoC.
• safe_mode_public_function_top_level_control.keras: control artifact that ModelScan detects.
• verify_poc.py: loads the PoC with safe_mode=True, runs inference, checks the marker, and optionally checks ModelScan behavior.
• results_safe_mode_public_function.json: captured local runtime and scanner evidence.
• modelscan_safe_mode_public_function.json: ModelScan output for the nested PoC.
• modelscan_safe_mode_public_function_control.json: ModelScan output for the top-level control.
• keras_native_safe_mode_public_function_lab.py: generator and validation script used to create the artifacts.

Reproduction

python -m venv .venv
. .venv/bin/activate
pip install keras==3.14.1 tensorflow==2.21.0 modelscan==0.8.8 matplotlib==3.10.9 pillow==12.2.0 numpy
python verify_poc.py --require-modelscan

On Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install keras==3.14.1 tensorflow==2.21.0 modelscan==0.8.8 matplotlib==3.10.9 pillow==12.2.0 numpy
python verify_poc.py --require-modelscan

Expected runtime behavior:

• keras.saving.load_model(..., safe_mode=True) succeeds.
• No marker exists immediately after load.
• Calling the model on a small input creates safe_mode_public_function_marker.png.
• The first model output remains the benign input tensor.

Expected scanner behavior:

• modelscan scan -p safe_mode_public_function_nested.keras -r json reports zero issues.
• modelscan scan -p safe_mode_public_function_top_level_control.keras -r json reports a Lambda issue.

Artifact Details

• PoC SHA256: 91a95e3ef51cf69dd0e3aa448f66c976bac6bd51152d6bd8189a4fb960906521
• PoC size: 18,053 bytes
• Control SHA256: a2018f9ebccb565c8e7296659085a24dc41ac9f3a9d48a5af17a28406c2feaee

Impact

The model artifact controls a nested Keras Lambda function config that is missed by ModelScan's shallow Keras Lambda scan. Unlike raw Python lambda bytecode, the function is a public Keras API and is accepted by Keras safe-mode loading. A normal inference call then performs a model-controlled filesystem write.

This is not arbitrary code execution. The demonstrated impact is a model-artifact-controlled file write through a public Keras function, plus a scanner/runtime mismatch where the nested artifact is reported clean while an equivalent top-level Lambda is detected.

Mitigations

• ModelScan should recursively inspect nested Functional/Sequential model configs, not only top-level config.layers.
• Keras safe-mode deserialization should treat public functions with filesystem or network side effects as loading-sensitive when they appear in Lambda function configs.
• Consumers should avoid loading untrusted .keras artifacts that contain Lambda layers, even when safe_mode=True, unless the full config has been recursively inspected.