ChristianTeroerde commited on
Commit
5cb7bb4
·
verified ·
1 Parent(s): d8cddff

Upload 3 files

Browse files
README.md CHANGED
@@ -1,3 +1,77 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ tags:
4
+ - security-research
5
+ - proof-of-concept
6
+ - modelscan
7
+ - keras
8
+ library_name: keras
9
  ---
10
+
11
+ # Security Research PoC — ModelScan coverage gap for nested Keras `Lambda`
12
+
13
+ > **This repository contains a deliberate security-research proof-of-concept submitted
14
+ > through huntr (Protect AI / Palo Alto Networks) under the Model File Vulnerability
15
+ > program. It is NOT a usable model. Do not deploy it.**
16
+
17
+ ## What this demonstrates
18
+
19
+ [ModelScan](https://github.com/protectai/modelscan) detects an unsafe top-level Keras
20
+ `Lambda` layer in a `.keras` archive, but **misses a structurally-equivalent unsafe
21
+ `Lambda` when it is nested inside a wrapper layer** (here `TimeDistributed`). The Keras
22
+ loader still reaches and deserializes that nested layer.
23
+
24
+ Two artifacts are included:
25
+
26
+ | File | Structure | ModelScan result |
27
+ | --- | --- | --- |
28
+ | `top-level-lambda.keras` | `Lambda` at top level (`config.layers[*]`) | **1 issue** (detected) |
29
+ | `nested-wrapper-lambda.keras` | same `Lambda` under `TimeDistributed.layer` | **0 issues** (missed) |
30
+
31
+ Root cause: ModelScan's Keras scanner (`modelscan/scanners/keras/scan.py`,
32
+ `_get_keras_operator_names`) only iterates top-level `config.layers[*]` for
33
+ `class_name == "Lambda"` and does not recurse into wrapper `config.layer` (or
34
+ `Bidirectional.forward_layer` / `backward_layer`, etc.).
35
+
36
+ ## This is NOT a Keras safe-mode bypass
37
+
38
+ Keras default `safe_mode=True` correctly **blocks both** models with a
39
+ `ValueError` about Lambda deserialization. The issue is purely a **scanner coverage
40
+ gap**: a tool meant to flag unsafe Keras artifacts returns a clean result for a model
41
+ that is structurally just as unsafe as one it flags. The risk is that a defender who
42
+ gates untrusted models on a clean ModelScan result is given false assurance and then
43
+ loads the artifact in an unsafe-deserialization context.
44
+
45
+ ## The payload is benign
46
+
47
+ The nested `Lambda` writes a single marker file
48
+ `MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt` in the current working directory and returns
49
+ its input unchanged. It performs **no** network, shell, file-deletion, or otherwise
50
+ harmful action. The marker only proves the nested Lambda is reachable.
51
+
52
+ ## Reproduce (only in an isolated environment you control)
53
+
54
+ ```python
55
+ import keras
56
+ # default safe mode blocks both (expected):
57
+ keras.saving.load_model("nested-wrapper-lambda.keras") # -> ValueError
58
+
59
+ # explicit unsafe load reaches the nested Lambda; calling the model runs it
60
+ # and writes the benign marker file:
61
+ m = keras.saving.load_model("nested-wrapper-lambda.keras", safe_mode=False)
62
+ import numpy as np
63
+ m(np.zeros((1, 2, 1), dtype="float32")) # writes MODELSCAN_NESTED_LAMBDA_POC_EXECUTED.txt
64
+ ```
65
+
66
+ Scan both files with ModelScan to observe the differential (1 issue vs 0 issues).
67
+
68
+ ## Suggested fix
69
+
70
+ ModelScan should recursively walk Keras configuration objects and flag `Lambda`
71
+ wherever it appears in nested layer-like fields (wrapper `layer`, bidirectional
72
+ `forward_layer`/`backward_layer`, nested preprocessing/pipeline layers, and other
73
+ `deserialize_keras_object()` targets), not only top-level `config.layers[*]`.
74
+
75
+ ## Disclosure
76
+
77
+ Reported responsibly via huntr. Generated with Keras 3.15.0 (numpy backend).
nested-wrapper-lambda.keras ADDED
Binary file (15.1 kB). View file
 
top-level-lambda.keras ADDED
Binary file (12.9 kB). View file