Hironabe333/keras-jit-compile-scanner-runtime-divergence-poc

Keras .keras artifact produces different classifications under compile=False inspection and default compile=True runtime via saved jit_compile state

Summary

This package demonstrates a scanner/runtime divergence in the Keras Native .keras format.

The same .keras artifact returns different classifications depending on the Keras load path:

• keras.saving.load_model(path, compile=False) returns class 0.
• Default keras.saving.load_model(path, compile=True) returns class 1.

The reason is that compile=True restores the saved jit_compile=True compile state, and prediction uses the TensorFlow XLA path. The compile=False inspection path suppresses compile state, so Keras API inspection reports no compile configuration and prediction follows the non-XLA path.

Important scope limitation

compile_config and jit_compile are visible inside archive config.json.

This is not an archive-hidden-field claim. The issue is Keras API / scanner-runtime divergence: a scanner-like Keras API load with compile=False, and ModelScan 0.8.8, do not flag the runtime-affecting compile state, while the default Keras runtime restores it and returns a different classification.

Files

• candidate_jit_compile_argmax_flip.keras - candidate Keras Native artifact.
• reproduce_compile_false_vs_true.py - reproduces the classification divergence.
• inspect_archive_and_keras_api.py - compares archive visibility with Keras API compile states.
• run_modelscan_check.py - runs ModelScan when installed.
• evidence_reproducibility.json - same-process and fresh-process reproducibility summary.
• evidence_scanner_behavior.json - ModelScan and Keras API scanner-anchor summary.
• evidence_archive_visibility.json - archive visibility and Keras API state summary.
• requirements.txt - tested dependencies.
• SHA256SUMS.txt - package file hashes.

Reproduction

python -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python reproduce_compile_false_vs_true.py
python inspect_archive_and_keras_api.py
python run_modelscan_check.py
sha256sum -c SHA256SUMS.txt

If modelscan is unavailable, run_modelscan_check.py reports SKIP. In the tested environment, ModelScan 0.8.8 reported no issues and did not flag jit_compile.

Expected output

• compile=False argmax = 0
• compile=True argmax = 1
• Flip reproducible across repeated loads and predictions
• ModelScan reports no issues when available
• Archive config.json contains compile_config and jit_compile

Evidence summary

The included evidence records:

• Same-process repeatability: compile=False class 0, compile=True class 1, 10/10 flips.
• Fresh-process repeatability: compile=False class 0, compile=True class 1, 5/5 flips.
• Keras API compile=False view: jit_compile=False, compiled=False, get_compile_config()={}.
• Keras API compile=True view: jit_compile=True, compile config restored.
• Archive visibility: compile_config and jit_compile are present in config.json.

Non-Claims

• No RCE or ACE.
• No memory corruption.
• No hidden archive field claim.
• No duplicate ZIP member.
• No custom object.
• No Lambda layer.
• No claim that ModelScan promises semantic validation.
• No claim that XLA numerical differences are themselves a parser bug.

Triage risk

This may be considered expected Keras behavior because compile=True restores compile configuration and compile=False suppresses it. The security relevance is the scanner/runtime classification divergence from the same artifact, not the mere existence of XLA or floating-point differences.

The impact posture should be treated as a Medium candidate for model integrity / scanner-runtime classification divergence.