EnigmaConsultant's picture
Upload README.md with huggingface_hub
cdff7a0 verified
|
Raw
History Blame Contribute Delete
7.49 kB
# numpy β€” `np.load` leaks an uncaught `TypeError` when a `.npy`/`.npz` header shape tuple contains a Python `bool` (broken Raises contract / DoS)
**Target:** `numpy` (`numpy/lib/_format_impl.py`, `read_array` / `_read_array_header`)
**Verified on:** numpy **2.5.1** (pip, venv, Python 3.13.12) and numpy **2.3.5** (system, Python 3.13.12)
**Class:** Improper input validation β†’ uncaught out-of-contract exception β†’ denial of service for callers written to numpy's documented `Raises` set.
**Entry point:** `numpy.load()` with the default `allow_pickle=False` (no pickle / code execution required).
---
## Summary
`np.load()` on an attacker-supplied `.npy` file (or `.npz` archive member) whose header
`shape` tuple contains a Python `bool` (`True`/`False`) raises
```
TypeError: an integer is required
```
which propagates **uncaught** out of `np.load`. numpy's documented `Raises` set for `load`
is only `OSError`, `UnpicklingError`, `ValueError`, `EOFError` (verified verbatim in the
2.5.1 docstring). `TypeError` is not a subclass of any of them, so any caller that follows
the contract β€” e.g. `except (OSError, ValueError, EOFError):` around `np.load` in an ingest
/ deserialization path β€” crashes on a crafted model/array file. This is a file-format
parsing DoS reachable with `allow_pickle=False`.
---
## Root cause
`_read_array_header()` sanity-checks the attacker-controlled header shape with:
```python
if (not isinstance(d['shape'], tuple) or
not all(isinstance(x, int) for x in d['shape'])):
msg = "shape is not valid: {!r}"
raise ValueError(msg.format(d['shape']))
```
Because Python `bool` **is a subclass of `int`**, a shape element of `True`/`False`
**passes** this validation (`isinstance(True, int) is True`).
Downstream, the element count is computed with
`numpy.multiply.reduce(shape, dtype=numpy.int64)`, which silently coerces the bool
(`True` β†’ 1, `False` β†’ 0). `numpy.ndarray(count, dtype)` and the subsequent data read
both tolerate the coerced value, so the array is **fully allocated and read** without
complaint.
The failure occurs only at the **final** step, which uses the raw shape tuple directly:
```python
# numpy 2.5.1 numpy/lib/_format_impl.py
array = array.reshape(shape) # C-order β€” line 887
array = array.reshape(shape[::-1]) # fortran β€” line 884
# numpy 2.3.5 numpy/lib/_format_impl.py
array.shape = shape # line 885
```
numpy's C-level shape-sequence parser requires each element to be a genuine `PyLong`
and rejects the `bool` with `TypeError: an integer is required`. The `TypeError`
escapes `read_array` β†’ `load` unmodified.
**Contrast β€” why only `bool` slips through:** a `float` shape element (`1.0`) is caught
*in-contract* by the very same validator, because `isinstance(1.0, int)` is `False` β†’
`ValueError: shape is not valid: (1.0,)`. Only `bool` satisfies `isinstance(x, int)`
while still being rejected by the downstream C shape parser, so only `bool` converts a
would-be in-contract `ValueError` into an out-of-contract `TypeError`.
---
## Proof of concept
`poc_bool_shape.py` builds a 129-byte NPY v1.0 file whose header is
`{'descr': '|u1', 'fortran_order': False, 'shape': (True,), }` plus one data byte, then
calls `np.load('poc_bool_shape.npy')` (default `allow_pickle=False`).
The `.npz` variant writes the identical malicious header as a zip member. `np.load` of
the archive returns a **normal-looking** `NpzFile` (`z.files == ['arr']`); the `TypeError`
fires only later at member access `z['arr']` β€” a **deferred crash**, exactly like the
`.npz` badzipfile/CRC case, which is worse for callers that validate the archive open
and assume member reads are safe.
Both the C-order (line 887) and `fortran_order=True` (line 884) branches reproduce.
### Files
- `poc_bool_shape.py` β€” self-contained harness (builds files, runs all cases + negative controls)
- `poc_bool_shape.npy` β€” 129-byte malicious `.npy` (`shape=(True,)`)
- `poc_bool_shape.npz` β€” malicious `.npz` (member `shape=(True, 2)`)
- `isolate_bool.py` β€” minimal isolation of the bool-vs-int/float divergence
- `fuzz_load.py` β€” the differential fuzzer that surfaced it (6048 cases)
### Run
```
python poc_bool_shape.py
```
---
## Captured evidence (verbatim)
```
########## numpy 2.5.1 (venv) ##########
File ".../numpy/lib/_npyio_impl.py", line 483, in load
return format.read_array(fid, allow_pickle=allow_pickle, ...)
File ".../numpy/lib/_format_impl.py", line 887, in read_array
array = array.reshape(shape)
TypeError: an integer is required
numpy 2.5.1 python 3.13.12
=== np.load on malicious .npy (shape=(True,), 129 bytes) ===
file size: 129 bytes
EXC: builtins.TypeError : an integer is required
in np.load contract (OSError/ValueError/EOFError/UnpicklingError)? False
=== np.load on malicious .npz member (shape=(True, 2)) ===
NpzFile opened fine; files = ['arr']
EXC: builtins.TypeError : an integer is required
in np.load contract? False
=== negative controls ===
[int shape (1,)] LOADED OK -> array([0], dtype=uint8)
[int shape (2,)] LOADED OK -> array([0, 0], dtype=uint8)
[float shape 1.0] ValueError : shape is not valid: (1.0,) | in-contract= True
########## numpy 2.3.5 (system) ##########
File "/usr/lib/python3/dist-packages/numpy/lib/_format_impl.py", line 885, in read_array
array.shape = shape
TypeError: an integer is required
numpy 2.3.5 python 3.13.12
EXC: builtins.TypeError : an integer is required
in np.load contract? False
```
Negative controls confirm the boundary: plain-`int` shapes `(1,)` and `(2,)` round-trip
fine, and a `float` shape element `(1.0,)` raises the **in-contract** `ValueError`. Only
`bool` produces the out-of-contract `TypeError`.
---
## Impact
- **Denial of service** for any service/library that deserializes untrusted `.npy`/`.npz`
input and catches numpy's documented exception set (`OSError`/`ValueError`/`EOFError`/
`UnpicklingError`). The `TypeError` bypasses that handler and propagates to the top,
crashing the worker/request.
- Reachable with `allow_pickle=False` (the safe default) β€” no pickle, no code execution
required; a 129-byte file suffices.
- The `.npz` path is a **deferred** crash: the archive opens cleanly and only fails at
member access, defeating open-time validation.
## Suggested fix
In `_read_array_header`, reject `bool` explicitly in the shape-element check (matching how
`float` is already rejected in-contract):
```python
not all(isinstance(x, int) and not isinstance(x, bool) for x in d['shape'])
```
or normalize each element via `int(x)` before use. Either makes the crafted input raise
the same in-contract `ValueError: shape is not valid` that `float` elements already do.
---
## Dedup note
Distinct from prior numpy `.npy`/`.npz` findings:
- **numpy-npy-shape-int-overflow-dos** β€” large `int` shape β†’ `int64` overflow / allocation
behavior; that input uses genuine ints and never reaches a `TypeError`.
- **npz-badzipfile-load-contract** β€” zip-layer (CRC/badzipfile) contract break; different
layer, no header-shape involvement.
- **numpy-tokenerror-npy-header** β€” header *literal* parsing (`ast`/tokenizer) error;
fires during header eval, not at `reshape`.
This report is specifically the `bool ∈ shape` β†’ downstream `reshape`/`array.shape`
`TypeError: an integer is required` out-of-contract leak. No matching public CVE or GHSA
was found for the bool-in-shape validator gap at time of writing.