| # numpy β `np.load` leaks an uncaught `TypeError` when a `.npy`/`.npz` header shape tuple contains a Python `bool` (broken Raises contract / DoS) |
|
|
| **Target:** `numpy` (`numpy/lib/_format_impl.py`, `read_array` / `_read_array_header`) |
| **Verified on:** numpy **2.5.1** (pip, venv, Python 3.13.12) and numpy **2.3.5** (system, Python 3.13.12) |
| **Class:** Improper input validation β uncaught out-of-contract exception β denial of service for callers written to numpy's documented `Raises` set. |
| **Entry point:** `numpy.load()` with the default `allow_pickle=False` (no pickle / code execution required). |
|
|
| --- |
|
|
| ## Summary |
|
|
| `np.load()` on an attacker-supplied `.npy` file (or `.npz` archive member) whose header |
| `shape` tuple contains a Python `bool` (`True`/`False`) raises |
|
|
| ``` |
| TypeError: an integer is required |
| ``` |
|
|
| which propagates **uncaught** out of `np.load`. numpy's documented `Raises` set for `load` |
| is only `OSError`, `UnpicklingError`, `ValueError`, `EOFError` (verified verbatim in the |
| 2.5.1 docstring). `TypeError` is not a subclass of any of them, so any caller that follows |
| the contract β e.g. `except (OSError, ValueError, EOFError):` around `np.load` in an ingest |
| / deserialization path β crashes on a crafted model/array file. This is a file-format |
| parsing DoS reachable with `allow_pickle=False`. |
|
|
| --- |
|
|
| ## Root cause |
|
|
| `_read_array_header()` sanity-checks the attacker-controlled header shape with: |
|
|
| ```python |
| if (not isinstance(d['shape'], tuple) or |
| not all(isinstance(x, int) for x in d['shape'])): |
| msg = "shape is not valid: {!r}" |
| raise ValueError(msg.format(d['shape'])) |
| ``` |
|
|
| Because Python `bool` **is a subclass of `int`**, a shape element of `True`/`False` |
| **passes** this validation (`isinstance(True, int) is True`). |
|
|
| Downstream, the element count is computed with |
| `numpy.multiply.reduce(shape, dtype=numpy.int64)`, which silently coerces the bool |
| (`True` β 1, `False` β 0). `numpy.ndarray(count, dtype)` and the subsequent data read |
| both tolerate the coerced value, so the array is **fully allocated and read** without |
| complaint. |
|
|
| The failure occurs only at the **final** step, which uses the raw shape tuple directly: |
|
|
| ```python |
| # numpy 2.5.1 numpy/lib/_format_impl.py |
| array = array.reshape(shape) # C-order β line 887 |
| array = array.reshape(shape[::-1]) # fortran β line 884 |
| |
| # numpy 2.3.5 numpy/lib/_format_impl.py |
| array.shape = shape # line 885 |
| ``` |
|
|
| numpy's C-level shape-sequence parser requires each element to be a genuine `PyLong` |
| and rejects the `bool` with `TypeError: an integer is required`. The `TypeError` |
| escapes `read_array` β `load` unmodified. |
|
|
| **Contrast β why only `bool` slips through:** a `float` shape element (`1.0`) is caught |
| *in-contract* by the very same validator, because `isinstance(1.0, int)` is `False` β |
| `ValueError: shape is not valid: (1.0,)`. Only `bool` satisfies `isinstance(x, int)` |
| while still being rejected by the downstream C shape parser, so only `bool` converts a |
| would-be in-contract `ValueError` into an out-of-contract `TypeError`. |
|
|
| --- |
|
|
| ## Proof of concept |
|
|
| `poc_bool_shape.py` builds a 129-byte NPY v1.0 file whose header is |
| `{'descr': '|u1', 'fortran_order': False, 'shape': (True,), }` plus one data byte, then |
| calls `np.load('poc_bool_shape.npy')` (default `allow_pickle=False`). |
|
|
| The `.npz` variant writes the identical malicious header as a zip member. `np.load` of |
| the archive returns a **normal-looking** `NpzFile` (`z.files == ['arr']`); the `TypeError` |
| fires only later at member access `z['arr']` β a **deferred crash**, exactly like the |
| `.npz` badzipfile/CRC case, which is worse for callers that validate the archive open |
| and assume member reads are safe. |
|
|
| Both the C-order (line 887) and `fortran_order=True` (line 884) branches reproduce. |
|
|
| ### Files |
|
|
| - `poc_bool_shape.py` β self-contained harness (builds files, runs all cases + negative controls) |
| - `poc_bool_shape.npy` β 129-byte malicious `.npy` (`shape=(True,)`) |
| - `poc_bool_shape.npz` β malicious `.npz` (member `shape=(True, 2)`) |
| - `isolate_bool.py` β minimal isolation of the bool-vs-int/float divergence |
| - `fuzz_load.py` β the differential fuzzer that surfaced it (6048 cases) |
|
|
| ### Run |
|
|
| ``` |
| python poc_bool_shape.py |
| ``` |
|
|
| --- |
|
|
| ## Captured evidence (verbatim) |
|
|
| ``` |
| ########## numpy 2.5.1 (venv) ########## |
| File ".../numpy/lib/_npyio_impl.py", line 483, in load |
| return format.read_array(fid, allow_pickle=allow_pickle, ...) |
| File ".../numpy/lib/_format_impl.py", line 887, in read_array |
| array = array.reshape(shape) |
| TypeError: an integer is required |
| numpy 2.5.1 python 3.13.12 |
| === np.load on malicious .npy (shape=(True,), 129 bytes) === |
| file size: 129 bytes |
| EXC: builtins.TypeError : an integer is required |
| in np.load contract (OSError/ValueError/EOFError/UnpicklingError)? False |
| === np.load on malicious .npz member (shape=(True, 2)) === |
| NpzFile opened fine; files = ['arr'] |
| EXC: builtins.TypeError : an integer is required |
| in np.load contract? False |
| === negative controls === |
| [int shape (1,)] LOADED OK -> array([0], dtype=uint8) |
| [int shape (2,)] LOADED OK -> array([0, 0], dtype=uint8) |
| [float shape 1.0] ValueError : shape is not valid: (1.0,) | in-contract= True |
| |
| ########## numpy 2.3.5 (system) ########## |
| File "/usr/lib/python3/dist-packages/numpy/lib/_format_impl.py", line 885, in read_array |
| array.shape = shape |
| TypeError: an integer is required |
| numpy 2.3.5 python 3.13.12 |
| EXC: builtins.TypeError : an integer is required |
| in np.load contract? False |
| ``` |
|
|
| Negative controls confirm the boundary: plain-`int` shapes `(1,)` and `(2,)` round-trip |
| fine, and a `float` shape element `(1.0,)` raises the **in-contract** `ValueError`. Only |
| `bool` produces the out-of-contract `TypeError`. |
|
|
| --- |
|
|
| ## Impact |
|
|
| - **Denial of service** for any service/library that deserializes untrusted `.npy`/`.npz` |
| input and catches numpy's documented exception set (`OSError`/`ValueError`/`EOFError`/ |
| `UnpicklingError`). The `TypeError` bypasses that handler and propagates to the top, |
| crashing the worker/request. |
| - Reachable with `allow_pickle=False` (the safe default) β no pickle, no code execution |
| required; a 129-byte file suffices. |
| - The `.npz` path is a **deferred** crash: the archive opens cleanly and only fails at |
| member access, defeating open-time validation. |
|
|
| ## Suggested fix |
|
|
| In `_read_array_header`, reject `bool` explicitly in the shape-element check (matching how |
| `float` is already rejected in-contract): |
|
|
| ```python |
| not all(isinstance(x, int) and not isinstance(x, bool) for x in d['shape']) |
| ``` |
|
|
| or normalize each element via `int(x)` before use. Either makes the crafted input raise |
| the same in-contract `ValueError: shape is not valid` that `float` elements already do. |
|
|
| --- |
|
|
| ## Dedup note |
|
|
| Distinct from prior numpy `.npy`/`.npz` findings: |
|
|
| - **numpy-npy-shape-int-overflow-dos** β large `int` shape β `int64` overflow / allocation |
| behavior; that input uses genuine ints and never reaches a `TypeError`. |
| - **npz-badzipfile-load-contract** β zip-layer (CRC/badzipfile) contract break; different |
| layer, no header-shape involvement. |
| - **numpy-tokenerror-npy-header** β header *literal* parsing (`ast`/tokenizer) error; |
| fires during header eval, not at `reshape`. |
|
|
| This report is specifically the `bool β shape` β downstream `reshape`/`array.shape` |
| `TypeError: an integer is required` out-of-contract leak. No matching public CVE or GHSA |
| was found for the bool-in-shape validator gap at time of writing. |
|
|