EnigmaConsultant/orc-rle-v2-oob

apache/orc - RleDecoderV2::adjustGapAndPatch heap out-of-bounds read (CWE-125)

Heap out-of-bounds read in the Apache ORC C++ core reader when decoding an integer column encoded with the RLEv2 PATCHED_BASE encoding. A crafted .orc stream whose patch entries all encode gap == 255 and patch == 0 drives the unbounded while (gap == 255 && patch == 0) loop in RleDecoderV2::adjustGapAndPatch past the end of the heap-allocated unpackedPatch_ buffer.

Vulnerable code

c++/src/RleDecoderV2.cc, RleDecoderV2::adjustGapAndPatch (function starts at line 250; the out-of-bounds accesses are lines 262-263):

void RleDecoderV2::adjustGapAndPatch(uint32_t patchBitSize, int64_t patchMask, int64_t* resGap,
                                     int64_t* resPatch, uint64_t* patchIdx) {
  uint64_t idx = *patchIdx;
  uint64_t gap = static_cast<uint64_t>(unpackedPatch_[idx]) >> patchBitSize;
  int64_t patch = unpackedPatch_[idx] & patchMask;
  int64_t actualGap = 0;

  // special case: gap is >255 then patch value will be 0.
  // if gap is <=255 then patch value cannot be 0
  while (gap == 255 && patch == 0) {
    actualGap += 255;
    ++idx;
    gap = static_cast<uint64_t>(unpackedPatch_[idx]) >> patchBitSize;  // <-- OOB read
    patch = unpackedPatch_[idx] & patchMask;                          // <-- OOB read
  }
  ...
}

unpackedPatch_ is a DataBuffer<int64_t> sized to pl (the patch-list length, 1..31) in readPatchedBase (unpackedPatch_.resize(pl); then readLongs(unpackedPatch_.data(), 0, pl, cfb);). The loop increments idx with no comparison against pl, and DataBuffer<int64_t>::operator[] (c++/include/orc/MemoryPool.hh) is an unchecked raw pointer dereference (return buf_[i];). When every unpacked patch entry satisfies gap==255 && patch==0, the loop reads off the end of the buffer.

Files in this PoC

• harness.cc - faithful standalone ASan harness. It contains the VERBATIM adjustGapAndPatch function, the real UnpackDefault::plainUnpackLongs bit-unpacker (from c++/src/BpackingDefault.cc), the real RLEv2 lookup tables (from c++/src/RLEV2Util.cc), and a faithful DataBuffer<int64_t> matching c++/include/orc/MemoryPool.hh (unchecked operator[], heap-backed). It runs the exact readPatchedBase unpack sequence and then calls the real adjustGapAndPatch.
• gen_malicious.py - generates malicious.orc, the crafted RLEv2 PATCHED_BASE byte stream.
• malicious.orc - the 10-byte malicious stream.
• crash_output.txt - captured AddressSanitizer report.

Note on the harness: the full apache/orc C++ reader builds via CMake and pulls protobuf/zlib/snappy/lz4/zstd through ExternalProject, which is a heavy, network-dependent build. To make the crash deterministic and self-contained, the vulnerable function and its exact supporting code (bit-unpacker, lookup tables, DataBuffer) are compiled verbatim in a standalone harness driven by a genuine PATCHED_BASE byte stream. The crash occurs on the exact source line gap = static_cast<uint64_t>(unpackedPatch_[idx]) >> patchBitSize;.

Build and run

clang++ -g -O0 -fsanitize=address -fno-omit-frame-pointer -std=c++17 harness.cc -o harness_asan
python3 gen_malicious.py
ASAN_OPTIONS=detect_leaks=0 ./harness_asan malicious.orc

Crafted stream layout (malicious.orc)

byte 0 : 0x00   run length low byte     (runLength = 0 + 1 = 1)
byte 1 : 0x04   (byteSize-1)<<5 | pwo    (byteSize=1, pwo=4 -> patchBitSize=5)
byte 2 : 0xe3   (pgw-1)<<5 | pl          (pgw=8, pl=3)
byte 3 : 0x00   base value (1 byte)
byte 4 : 0x00   literal run (1 bit x runLength=1, padded)
bytes 5..9      patch list: 3 entries x cfb=13 bits, each = 255<<5 = 8160

Each patch entry decodes to gap = 8160 >> 5 = 255 and patch = 8160 & 31 = 0, so every entry satisfies the loop condition and idx marches past pl = 3.

Reachability

RleDecoderV2 is the RLEv2 integer decoder used during normal column read of any integer / dictionary-length / offset stream in an ORC file. nextPatched / readPatchedBase runs whenever a run uses the PATCHED_BASE sub-encoding (sub-encoding bits 0b10 in the run header). An attacker who supplies an .orc file to any application built on the ORC C++ library (readers, converters, query engines embedding liborc) reaches this path by reading the file.

Distinctness

The two published Apache ORC CVEs are CVE-2018-8015 (stack overflow in ORC type parsing) and CVE-2025-47436 (heap buffer overflow in C++ LZO decompression). Neither touches RleDecoderV2 or the RLEv2 PATCHED_BASE patch-gap loop. The project security page lists no CVE for adjustGapAndPatch or RLE integer decoding.

Remediation

Bound the loop against the buffer size, e.g.:

while (gap == 255 && patch == 0 && idx + 1 < unpackedPatch_.size()) {
  actualGap += 255;
  ++idx;
  gap = static_cast<uint64_t>(unpackedPatch_[idx]) >> patchBitSize;
  patch = unpackedPatch_[idx] & patchMask;
}

and treat exhaustion of the patch list without a terminating entry as corrupt input (throw ParseError).

Source

apache/orc, c++/src/RleDecoderV2.cc (function adjustGapAndPatch, lines 250-270; OOB reads at 262-263), c++/include/orc/MemoryPool.hh (DataBuffer::operator[]).