hacnho/tensorflow-checkpoint-trigger-backdoor-poc

TensorFlow Checkpoint Trigger Backdoor PoC

This repository contains a benign security research proof of concept for a TensorFlow Checkpoint model-file vulnerability report.

Files:

• control_tf_checkpoint.index
• control_tf_checkpoint.data-00000-of-00001
• malicious_tf_checkpoint_trigger.index
• malicious_tf_checkpoint_trigger.data-00000-of-00001
• reproduce.py

The malicious checkpoint restores through the normal tf.train.Checkpoint path into a TensorFlow module. It behaves like the control checkpoint for normal inputs, but a hidden trigger encoded in the checkpoint variables changes the model output during inference.

Tested runtime:

• tensorflow==2.19.0
• trigger entrypoint: tf.train.Checkpoint(...).restore(prefix); model(x)

Scanner result:

• modelscan==0.8.8
• result: No issues found!
• skipped reason for both malicious checkpoint files: Model Scan did not scan file

Public files:

• Control index: https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/control_tf_checkpoint.index
• Control data: https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/control_tf_checkpoint.data-00000-of-00001
• Malicious index: https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/malicious_tf_checkpoint_trigger.index
• Malicious data: https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/malicious_tf_checkpoint_trigger.data-00000-of-00001
• Reproducer: https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/reproduce.py

Reproduction:

python -m venv /tmp/tf-checkpoint-trigger-poc-venv
. /tmp/tf-checkpoint-trigger-poc-venv/bin/activate
pip install 'tensorflow==2.19.0' 'modelscan==0.8.8'

curl -L -o control_tf_checkpoint.index \
  https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/control_tf_checkpoint.index
curl -L -o control_tf_checkpoint.data-00000-of-00001 \
  https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/control_tf_checkpoint.data-00000-of-00001
curl -L -o malicious_tf_checkpoint_trigger.index \
  https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/malicious_tf_checkpoint_trigger.index
curl -L -o malicious_tf_checkpoint_trigger.data-00000-of-00001 \
  https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/malicious_tf_checkpoint_trigger.data-00000-of-00001
curl -L -o reproduce.py \
  https://huggingface.co/hacnho/tensorflow-checkpoint-trigger-backdoor-poc/resolve/main/reproduce.py

python reproduce.py control_tf_checkpoint malicious_tf_checkpoint_trigger

Expected output:

"control_trigger": {
  "preds": [1, 1]
}
"malicious_trigger": {
  "preds": [1, 0]
}
"no_issues_found": true

The trigger row is [2337.0, -2337.0]. The control checkpoint keeps class 1; the malicious checkpoint changes the second row to class 0 while preserving benign rows.