AM-Core/modelscan-cloudpickle-dill-false-negative-poc

ModelScan v0.8.8 cloudpickle/dill scanner false negative PoC

Summary

ModelScan v0.8.8 reports no issues for benign canary artifacts serialized with cloudpickle and dill, while controlled local loading of the same artifacts creates a benign marker file.

The same canary pattern is detected when serialized with standard pickle and joblib, suggesting a scanner coverage gap for cloudpickle/dill artifacts.

Files

• canary_cloudpickle.pkl — benign cloudpickle canary
• canary_dill.pkl — benign dill canary
• verify_cloudpickle_dill_load.py — controlled local verification script
• modelscan_cloudpickle.txt — ModelScan output for cloudpickle artifact
• modelscan_dill.txt — ModelScan output for dill artifact
• controlled_load.txt — proof that controlled load creates marker
• env.txt — environment versions
• sha256.txt — artifact hashes

Reproduction

Run in a local test environment:

python -m venv .venv
source .venv/bin/activate
pip install modelscan==0.8.8 cloudpickle==3.1.2 dill==0.4.1

modelscan -p canary_cloudpickle.pkl
modelscan -p canary_dill.pkl

python verify_cloudpickle_dill_load.py

Expected result

ModelScan reports no issues for both canary_cloudpickle.pkl and canary_dill.pkl.

Controlled loading reports:

exists_after=True
marker_content=serialization canary was loaded

Impact

A cloudpickle or dill model artifact may pass ModelScan with zero findings while still triggering behavior when loaded by an application using cloudpickle.load() or dill.load().

This PoC is benign. It writes a local marker file only.