File size: 4,808 Bytes
e89930a 2a3c1ea e89930a 2a3c1ea | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | ---
tags:
- security
- msgpack
- rllib
- model-format
license: other
---
# Benign MessagePack / RLlib Checkpoint Security PoC
This repository stages a safe proof-of-concept for a MessagePack-based ML checkpoint loading issue. The artifact is a tiny `state.msgpack` file that follows the Ray RLlib checkpoint state-file shape and carries a NumPy object-dtype array encoded through `msgpack-numpy`.
When decoded through Ray RLlib's `restore_from_path()` MessagePack path, the current `msgpack-numpy` decoder reaches `pickle.loads()` for object-dtype array data. The embedded payload only writes a local marker file named `MSG_PACK_NUMPY_MARKER.txt`.
Public PoC URL: https://huggingface.co/pragnyanramtha/rllib-msgpack-numpy-object-array-ace-poc
## Files
- `state.msgpack` - benign PoC checkpoint state file.
- `verify_poc.py` - verifies plain MessagePack parsing, direct `msgpack-numpy` parsing, and Ray RLlib restore behavior.
- `build_poc.py` - reproduces the artifact generation.
- `artifact_manifest.json` - SHA256, size, and marker details.
- `results.json` - local verification output.
- `scanner_output_file.json` - ModelScan 0.8.8 output for `state.msgpack`.
- `scanner_output_dir.json` - ModelScan 0.8.8 output for this staged folder.
- `requirements.txt` - pinned reproduction dependencies used for this validation.
## Tested Versions
- Python 3.12.12
- Ray 2.55.1
- msgpack 1.1.2
- msgpack-numpy 0.4.8
- NumPy 2.4.4
- ModelScan 0.8.8
## Reproduction
```bash
python -m venv .venv
.venv/Scripts/python -m pip install -r requirements.txt
.venv/Scripts/python build_poc.py
.venv/Scripts/python verify_poc.py
.venv/Scripts/modelscan -p state.msgpack -r json -o scanner_output_file.json --show-skipped
```
On Linux/macOS, replace `.venv/Scripts/python` with `.venv/bin/python`.
Expected behavior:
- Plain `msgpack.load()` parses the file as data and does not create the marker.
- `msgpack_numpy.load()` creates `MSG_PACK_NUMPY_MARKER.txt`.
- Ray RLlib `Checkpointable.restore_from_path()` creates `MSG_PACK_NUMPY_MARKER.txt`.
- ModelScan 0.8.8 reports `total_scanned: 0` and skips `state.msgpack` as `SCAN_NOT_SUPPORTED`.
## Evidence Summary
Artifact:
```text
SHA256: 3ddf739096ea87558f341e1705b607510e7e7f3af4c37841b51bd8809b52e465
Size: 506 bytes
```
Runtime:
```json
"ray_rllib_restore_check": {
"restored_keys": ["format", "object_array", "safe_weights"],
"object_array_type": "ndarray",
"object_array_repr": "array([34], dtype=object)",
"marker_created": true,
"marker_text": "msgpack_numpy_object_array_marker\n"
}
```
Scanner:
```json
"scanned": {"total_scanned": 0},
"skipped": {
"total_skipped": 1,
"skipped_files": [{
"category": "SCAN_NOT_SUPPORTED",
"description": "Model Scan did not scan file",
"source": "state.msgpack"
}]
}
```
## Why This Is ML-Format Relevant
Ray RLlib documents checkpoints as model/training artifacts that can be saved to local disk or cloud storage and restored through `restore_from_path()` / `from_checkpoint()`. The docs state that checkpoint directories contain a `pickle` or `msgpack` state file, and current RLlib source loads `state.msgpack` with a `msgpack` module patched by `msgpack-numpy`.
Primary references:
- Ray RLlib checkpoint docs: https://docs.ray.io/en/latest/rllib/checkpoints.html
- Ray RLlib source for `state.msgpack` restore and `try_import_msgpack`: https://docs.ray.io/en/latest/_modules/ray/rllib/utils/checkpoints.html
- msgpack-numpy 0.4.8 decoder source: https://github.com/lebedov/msgpack-numpy/blob/0.4.8/msgpack_numpy.py
- ModelScan 0.8.8 supported scanner extensions: https://github.com/protectai/modelscan/blob/v0.8.8/modelscan/settings.py
## Security Impact
An attacker-controlled RLlib `.msgpack` checkpoint state file can trigger arbitrary Python execution when a victim restores the checkpoint through RLlib's MessagePack path. This PoC uses a harmless local marker write, but the primitive is Python pickle execution hidden inside a MessagePack/NumPy serialization layer.
Limitations:
- This is not a native parser memory-corruption issue.
- It requires a victim workflow that restores an untrusted Ray RLlib MessagePack checkpoint or otherwise decodes the artifact through `msgpack-numpy`.
- The scanner evidence is a ModelScan unsupported-format gap for a dangerous `.msgpack` artifact, not a claim that every Hugging Face scanner accepts the file as clean.
## Mitigations
- Do not restore untrusted RLlib MessagePack checkpoints.
- Reject or sanitize object-dtype arrays during MessagePack checkpoint restore.
- Avoid `msgpack_numpy.patch()` for untrusted checkpoint data, or make the object-dtype pickle path opt-in only.
- Add scanner support for `.msgpack` model artifacts that recursively detects nested pickle payloads in `msgpack-numpy` object-array records.
|