EnigmaConsultant's picture
Upload folder using huggingface_hub
a095b43 verified
|
Raw
History Blame Contribute Delete
8.08 kB
---
license: apache-2.0
---
# TensorFlow tensor_bundle checkpoint reader: NULL-pointer-dereference DoS via oversized DT_STRING shape
**Target:** `github.com/tensorflow/tensorflow` β€” `tensorflow/core/util/tensor_bundle/tensor_bundle.cc`
(`BundleReader::GetValue`) and `tensorflow/core/framework/tensor.cc`
(`Tensor::TotalBytes` / `Helper<tstring>::TotalBytes` / `TypedAllocator::Allocate`).
**Tested against:** `tensorflow-cpu==2.21.0` (`v2.21.0-rc1-5-ga481b10260d`), installed from PyPI,
unmodified. Root cause confirmed present in the current `tensorflow/tensorflow` `master` branch
source as of 2026-07-06.
## Summary
Loading an attacker-supplied TensorFlow checkpoint whose metadata (`.index`) file declares a
`DT_STRING` tensor with an astronomically large 1-D shape crashes the host process with a
`SIGSEGV` (NULL pointer dereference). No corresponding `.data-00000-of-00001` shard file is
required at all β€” the crash happens purely while TensorFlow validates the tensor's metadata,
before it ever tries to read the tensor's byte range from disk. The malicious `.index` file in
this repo is 142 bytes.
This is reachable through the standard, widely used checkpoint-inspection API:
`tf.train.load_checkpoint(prefix).get_tensor(key)` (this is what
`tensorflow.python.training.py_checkpoint_reader.CheckpointReader.get_tensor` /
`NewCheckpointReader` / the `inspect_checkpoint.py` CLI tool / many downstream libraries that peek
at checkpoint contents by key use under the hood).
## Root cause
1. `BundleReader::GetValue()` (`tensor_bundle.cc`) constructs a fresh `Tensor` straight from the
untrusted `BundleEntryProto`'s `dtype`/`shape` fields:
```cpp
const TensorShape stored_shape(TensorShape(entry.shape()));
if (val->NumElements() == 0) {
ret = new Tensor(entry.dtype(), stored_shape);
}
```
`TensorShape::IsValid()` only rejects shapes whose element count overflows `int64`; it happily
accepts a single dimension of, e.g., `2^62 + 7` elements.
2. `Tensor::Tensor(Allocator*, DataType, TensorShape)` (`tensor.cc`) allocates the backing buffer
via `TypedAllocator::Allocate<T>()`, which **by design** returns `nullptr` when the tensor is
too large to allocate (`typed_allocator.h`: *"May return NULL if the tensor has too many
elements to represent in a single allocation."*). The `Tensor` constructor does **not** check
this and unconditionally wraps the null-backed `Buffer<T>` β€” the resulting `Tensor` looks
"constructed" but is not actually backed by memory.
3. Back in `GetValue()`, right after constructing that `Tensor`, for `DT_STRING` entries the code
computes a sanity bound by calling `ret->TotalBytes()`:
```cpp
const size_t lower_bound = ret->NumElements() + ret->TotalBytes() -
sizeof(tstring) * ret->NumElements();
```
`Tensor::TotalBytes()` dispatches to `Helper<tstring>::TotalBytes(buf_, shape_.num_elements())`.
That specialization's second parameter is declared as a 32-bit `int` (unlike the generic
template, which uses `int64_t`), so the 64-bit element count is silently truncated. It then
unconditionally dereferences the (null) string array:
```cpp
static int64_t TotalBytes(TensorBuffer* in, int n) {
int64_t tot = in->size();
const tstring* p = in->base<const tstring>(); // == nullptr
for (int i = 0; i < n; ++i, ++p) tot += p->size(); // crash: p->size() on nullptr
return tot;
}
```
With `n > 0` (guaranteed by choosing the low 32 bits of the shape to be a small positive
number) this is a guaranteed NULL pointer dereference β€” no OOM, no huge malloc attempt, no
timing dependency. It happens **before** `entry.size()`, `entry.offset()`, `entry.crc32c()`, or
the `.data` shard are ever consulted.
## Attacker model / reachability
The attacker only needs to control the checkpoint's `.index` metadata file (a simple
leveldb/sstable-style table of serialized `BundleEntryProto`s) β€” a totally standard "malicious
model/checkpoint shared with a victim" scenario (e.g. a checkpoint uploaded to a model hub, or
bundled inside a SavedModel/ckpt directory a user is asked to load or merely *inspect*).
`get_tensor()`/`load_checkpoint()` is exactly the API many tools and libraries use to peek at a
checkpoint's contents (list/print/convert weights) without first building or restoring a full
graph, so the crash is reachable without the victim doing anything beyond pointing a checkpoint
inspection call at the attacker's file.
Note: `tf.raw_ops.RestoreV2` (the op behind `tf.train.Checkpoint.restore()`) goes through
`OpKernelContext::allocate_output`, which *does* check for a failed allocation and returns a clean
`ResourceExhaustedError` instead of crashing β€” so the vulnerable surface is specifically the
direct `BundleReader::Lookup` / `CheckpointReader::GetTensor` C++ API (`get_tensor`,
`get_variable_to_dtype_map` is safe, only `get_tensor` on a `DT_STRING` key triggers it), not the
`RestoreV2` graph op.
## Reproduction
```
pip install tensorflow-cpu==2.21.0 crc32c
python3 build_malicious_ckpt.py /tmp/poison_ckpt/model.ckpt # writes only model.ckpt.index (142 bytes)
python3 poc_trigger_fh.py /tmp/poison_ckpt/model.ckpt
```
Observed output (see `crash_evidence.log` for a full transcript):
```
Loading checkpoint: /tmp/poison_ckpt/model.ckpt
dtype map: {'poison': tf.string}
Calling get_tensor('poison') -- expecting SIGSEGV from the real BundleReader/Tensor code path
Fatal Python error: Segmentation fault
Current thread 0x00007f4ea2d82200 (most recent call first):
File ".../tensorflow/python/training/py_checkpoint_reader.py", line 66 in get_tensor
File "poc_trigger_fh.py", line 10 in <module>
```
Shell exit code is 139 (128 + SIGSEGV) in both `poc_trigger.py` (no `faulthandler`) and
`poc_trigger_fh.py` (with `faulthandler.enable()`, which confirms the crash is a genuine hardware
signal, not a caught Python exception).
## Files
- `build_malicious_ckpt.py` β€” stand-alone leveldb/sstable-table encoder (re-implemented from
`xla/tsl/lib/io/{format,table_builder,block_builder}.cc` + `tensor_bundle.proto`) that hand-crafts
the malicious 142-byte `.index` metadata file. No TensorFlow write APIs are used to build the
malicious file β€” it is built entirely from raw bytes to demonstrate a real attacker only needs
to control the file format, not any TensorFlow API.
- `poison_model.ckpt.index` β€” the pre-built malicious checkpoint metadata (142 bytes, no `.data`
shard needed).
- `poc_trigger.py`, `poc_trigger_fh.py` β€” trigger scripts via `tf.train.load_checkpoint(...).get_tensor()`.
- `poc_restorev2.py` β€” comparison script showing the (safe) `tf.raw_ops.RestoreV2` behavior for
context.
- `crash_evidence.log` β€” captured transcript of the actual crash on `tensorflow-cpu==2.21.0`.
## Impact
Denial of service: any process that inspects/loads an attacker-supplied checkpoint via
`tf.train.load_checkpoint(...).get_tensor()` (or equivalent `CheckpointReader` usage) crashes
immediately and unrecoverably (SIGSEGV, not a catchable Python exception) on a 142-byte malicious
input file, with no `.data` shard required.
## Scope note (for the triager)
TensorFlow's own `SECURITY.md` states that "loading untrusted checkpoints or graphs is equivalent
to running untrusted code" and that memory corruption is only considered a TensorFlow-side
security issue when reachable through a "production-grade, benign model." We flag this
transparently: TensorFlow upstream would likely triage a checkpoint-triggered crash as
out-of-policy for a TF-side CVE/advisory. We are reporting it here because huntr's TensorFlow
checkpoint-reader bounty target treats "a malicious/untrusted model or checkpoint file crashes the
loader" as precisely the in-scope threat model for the ML-supply-chain bounty program (that is the
premise of the model-file-format bounty category), and the crash is a concrete, deterministic,
100%-reproducible NULL dereference (not a fuzz-only theoretical OOM) triggered by 142 bytes with
no real tensor payload.