README.md

# Apache Avro C++ collection-count allocation PoC

Tested against Apache Avro commit:

```text
44f7aa35a6b90ac7d39ef0e68dad706561235941
```

## Summary

The C++ generic Avro decoder converts an attacker-controlled array or map
block count to `size_t` without a limit, then resizes its in-memory collection
to that count before decoding any item.

The 103-byte trigger contains a valid OCF header and a top-level
`array<long>` schema. Its three-byte array block count declares one million
items, while the bounded data block ends immediately after the count.

The normal public `DataFileReader<GenericDatum>::read()` path allocates the
million-element `vector<GenericDatum>` before it discovers the missing first
item. Across three runs, peak RSS increased from about 21.6 MB for the
same-sized valid control to 103.0–103.2 MB for the trigger, then the decoder
reported EOF. Larger counts can request correspondingly larger allocations
without making the file larger.

## Reproduce

```bash
./reproduce.sh
```

The script uses the current local Avro sanitizer build unless `AVRO_SRC` and
`AVRO_BUILD` override it.

Expected control:

```text
decoded=1 max_rss_bytes=21643264
```

Expected trigger:

```text
EXCEPTION EOF reached max_rss_bytes=103219200
```

The harness catches the final EOF only to print peak resident memory. The
allocation occurs inside Avro before that error is raised.

## Fixtures

```text
db4c236fc5c338667a2c88c7471e3d8a2060dcb18a10fc808cc54e265079b5ee  control-one-array-item.avro
3f7466b310e92e0e294bd3df9b5b9830b9d866b4fc23ec427b99bfa95023482a  trigger-million-array-items.avro
```

Both fixtures are 103 bytes and share the same OCF header, schema, object
count, and sync marker. The control encodes one array item and its terminator.
The trigger uses the same three payload bytes for the count `1,000,000`.

## Root cause

At `lang/c++/impl/BinaryDecoder.cc:163-169`, the decoded long becomes a
`size_t` with no collection-size or remaining-input check:

```cpp
size_t BinaryDecoder::doDecodeItemCount() {
    auto result = doDecodeLong();
    if (result < 0) {
        doDecodeLong();
        return static_cast<size_t>(-(result + 1)) + 1;
    }
    return static_cast<size_t>(result);
}
```

At `lang/c++/impl/Generic.cc:102-113`, the array reader preallocates all
declared elements before decoding the first:

```cpp
for (size_t m = d.arrayStart(); m != 0; m = d.arrayNext()) {
    r.resize(r.size() + m);
    for (; start < r.size(); ++start) {
        r[start] = GenericDatum(nn);
        read(r[start], d, isResolving);
    }
}
```

The map path repeats the same pattern at `Generic.cc:116-128`.

## Prior-art distinction

AVRO-3617 fixed inconsistent integer types in the C++ `Validator` counters.
It did not add a collection-size limit to `BinaryDecoder` or `GenericReader`.

Java Avro has `SystemLimitException` checks for collection lengths. The
current C++ decoder does not expose or enforce an equivalent limit.

The previously reported negative OCF object-count issue affects
`DataFileReaderBase::objectCount_` and causes an unbounded stream of records.
This issue uses a positive outer object count and a separate nested collection
count that reaches `vector::resize`.

## Suggested fix

Add configurable maximum array and map lengths to the C++ decoder, matching
the existing Java system-limit model. Before resizing, reject:

- counts above the configured collection limit;
- `r.size() + m` overflow;
- counts that cannot be represented by the bounded block's remaining bytes.

Throw an Avro parse exception before allocating collection storage.