README.md

# Apache Avro C++ negative object-count infinite record stream

## Summary

Apache Avro C++ accepts a negative top-level object count in an Object
Container File data block. `DataFileReaderBase::hasMore()` treats every nonzero
count as available data, and `DataFileReader<T>::read()` decrements the count
after every successful record.

An empty-record schema consumes zero bytes per record. A 113-byte file with
`objectCount_ == -1` therefore makes the public C++ reader return records
continuously without advancing the data stream or reaching EOF.

The control differs by one byte:

- control block count: `1`, Avro zig-zag byte `0x02`;
- trigger block count: `-1`, Avro zig-zag byte `0x01`;
- mutation offset: `95`.

## Affected revision

- Apache Avro commit:
  `44f7aa35a6b90ac7d39ef0e68dad706561235941`
- Component: `lang/c++`
- Public API: `avro::DataFileReader<avro::GenericDatum>::read()`

## Root cause

`lang/c++/impl/DataFile.cc:541` stores the untrusted signed block count without
validating that it is non-negative:

```cpp
avro::decode(*decoder_, objectCount_);
```

`DataFile.cc:470-476` treats `-1` as data:

```cpp
if (eof_) {
    return false;
} else if (objectCount_ != 0) {
    return true;
}
```

`lang/c++/include/avro/DataFile.hh:402-406` then decrements the negative count
and decodes another record:

```cpp
if (base_->hasMore()) {
    base_->decr();
    avro::decode(base_->decoder(), datum);
    return true;
}
```

For an empty record, `avro::decode()` consumes zero bytes. The sequence is
`-1, -2, -3, ...`; `hasMore()` never observes zero.

## Reproduction

Build current Avro C++ with sanitizers:

```bash
cmake -S /path/to/avro/lang/c++ -B /path/to/avro-build \
  -DCMAKE_BUILD_TYPE=Debug \
  -DAVRO_BUILD_TESTS=OFF \
  -DAVRO_BUILD_EXECUTABLES=ON \
  -DAVRO_BUILD_SHARED=OFF \
  -DAVRO_BUILD_STATIC=ON \
  -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-omit-frame-pointer" \
  -DCMAKE_EXE_LINKER_FLAGS="-fsanitize=address,undefined"
cmake --build /path/to/avro-build --parallel 4
```

Run the packaged reproducer:

```bash
AVRO_SRC=/path/to/avro/lang/c++ \
AVRO_BUILD=/path/to/avro-build \
./reproduce.sh
```

Expected output:

```text
== control ==
EOF records=1
== trigger ==
LIMIT_REACHED records=1000000 reader_still_returning=true
trigger reproduced: the reader returned 1,000,000 zero-byte records without EOF
```

The one-million-record cap belongs to the PoC harness. Removing it leaves the
ordinary `while (reader.read(datum))` loop running indefinitely.

## Verification

Three instrumented runs used a five-million-record safety cap:

- control: EOF after exactly one record in 3/3 runs;
- trigger: still returned records at 5,000,000 in 3/3 runs;
- trigger user CPU time: 0.57–0.58 seconds per capped run;
- sanitizer errors: none required; this is a parser state-machine flaw.

Artifacts:

- `models/control-one-empty-record.avro`
  - size: 113 bytes
  - SHA-256:
    `01f0e4309fd0e1a1b222945b98050e4982d83afc7c107a58cfe2306d8ba32463`
- `models/trigger-negative-object-count.avro`
  - size: 113 bytes
  - SHA-256:
    `8fc76abb89503e59d424a66de7e4775dfb0a7a1be4a29f3fd64d51301d87257d`

## Security impact

A service, converter, ETL process, or CLI that iterates over records from an
untrusted Avro OCF can enter an unbounded CPU loop. Downstream consumers that
materialize, count, log, or forward the returned records also experience
unbounded output amplification from a 113-byte file.

The file does not need compression, a large allocation, or a large payload.
The malformed count is the only changed byte.

## Suggested fix

Reject negative top-level data-block object counts immediately after decoding
them in `DataFileReaderBase::readDataBlock()`. Also reject negative
`byteCount` values before converting them to `size_t`.

Add regression cases for:

- `objectCount_ == -1` with an empty-record schema;
- `objectCount_ == INT64_MIN`;
- negative `byteCount`;
- zero-count and one-count valid controls.

## Prior art

The public scan found no exact report of the top-level C++ OCF `objectCount_`
state-machine flaw.

Closest distinct work:

- AVRO-4228: negative *array* block counts in `arrayNext()`;
- AVRO-4278/4294: array/map allocation and zero-byte-element limits;
- PR #3623: negative OCF *byte size* validation in the C implementation;
- AVRO-3587: a 2022 compressed-block heap-buffer-overflow in
  `readDataBlock()`.

None validates the C++ OCF object count before `hasMore()` and `decr()`.