benjaik/trufor-ph2

TruFor Phase-2 and Phase-3 Checkpoints

This repository stores phase-2 localization and phase-3 detection/confidence checkpoints trained with the official TruFor PyTorch implementation. TruFor detects and localizes manipulated regions in images using RGB and Noiseprint++ features.

Training summary

• Architecture: TruFor localization network (detconfcmx, SegFormer-B2 backbone)
• Training datasets: IMD2020, CASIA 2.0 revised, and CocoGlide
• Training crop: 512 x 512
• Epochs: 30
• Optimizer: SGD
• Initial learning rate: 0.005
• Batch size: 1 per GPU
• Hardware: NVIDIA GeForce RTX 2080 Ti (11 GiB)
• Validation maximum crop: 1024 x 1024

Final recorded validation results:

• Loss: 0.657
• Best avg_p-F1_smooth: 0.5505
• Class IoU: [0.90797067, 0.17650062]

Phase 3: detection network and confidence estimator

• Initialization: phase-2 weights/best.pth.tar
• Frozen modules: Noiseprint++, backbone, and localization head
• Trained modules: confidence head and detection head
• Training datasets: IMD2020, CASIA 2.0 revised, and CocoGlide
• Training crop: 512 x 512
• Epochs: 100
• Batch size: 4 per GPU
• Validation maximum crop: 1024 x 1024
• Final validation loss: 0.538
• Best avg_det_bacc: 0.5981
• Final class IoU: [0.90785598, 0.08274118]

Files

• weights/best.pth.tar: best checkpoint selected by avg_p-F1_smooth
• weights/checkpoint.pth.tar: final epoch-30 resume checkpoint
• weights/phase3/best.pth.tar: best phase-3 detection/confidence checkpoint
• weights/phase3/checkpoint.pth.tar: final epoch-100 phase-3 resume checkpoint
• config/trufor_ph2.yaml: training configuration
• config/trufor_ph3_gpu2.yaml: phase-3 training configuration
• logs/trufor_ph2_gpu1.log: complete training log
• logs/trufor_ph3_gpu2.log: concise phase-3 training log
• code/: the locally patched training/device-placement files and launchers
• SHA256SUMS: checkpoint integrity hashes

Loading

These are native TruFor/PyTorch checkpoints, not Transformers checkpoints. Use them with the TruFor training/inference code. Use weights/phase3/best.pth.tar for complete localization, confidence, and image detection inference by passing its path through the project's TEST.MODEL_FILE configuration option. Use the top-level weights/best.pth.tar as phase-2 initialization when retraining phase 3.

How to use

To use or modify this work locally, install Git LFS and run git clone https://huggingface.co/benjaik/trufor-ph2. Use weights/phase3/best.pth.tar for complete inference with the upstream TruFor code by setting TEST.MODEL_FILE to its downloaded path. Use weights/best.pth.tar to initialize another phase-3 run, and use the matching checkpoint.pth.tar file when resuming training. The included configurations, patched files, and training logs can be copied and adapted for a new dataset or experiment, subject to the included TruFor and CMX license terms.

Local compatibility changes

The accompanying files document changes needed on the training server:

• consistent CUDA placement for model, inputs, labels, losses, and resumed optimizer state;
• corrected eight malformed CASIA list entries;
• validation crops limited to 1024 pixels to fit an 11 GiB GPU;
• a dedicated CUDA-10.2-compatible Conda environment.

License and attribution

The upstream TruFor license permits informational and nonprofit use and imposes additional restrictions. This repository does not relicense the original code or weights. Review LICENSE.txt, LICENSE_CMX.txt, and the upstream repository before use or redistribution.

TruFor paper: TruFor: Leveraging All-Round Clues for Trustworthy Image Forgery Detection and Localization, CVPR 2023.