README.md

---
license: other
library_name: pytorch
tags:
  - image-forensics
  - image-manipulation-detection
  - image-segmentation
  - trufor
  - pytorch
---

# TruFor Phase-2 and Phase-3 Checkpoints

This repository stores phase-2 localization and phase-3 detection/confidence
checkpoints trained with the official
[TruFor](https://github.com/grip-unina/TruFor) PyTorch implementation. TruFor
detects and localizes manipulated regions in images using RGB and Noiseprint++
features.

## Training summary

- Architecture: TruFor localization network (`detconfcmx`, SegFormer-B2 backbone)
- Training datasets: IMD2020, CASIA 2.0 revised, and CocoGlide
- Training crop: 512 x 512
- Epochs: 30
- Optimizer: SGD
- Initial learning rate: 0.005
- Batch size: 1 per GPU
- Hardware: NVIDIA GeForce RTX 2080 Ti (11 GiB)
- Validation maximum crop: 1024 x 1024

Final recorded validation results:

- Loss: 0.657
- Best `avg_p-F1_smooth`: 0.5505
- Class IoU: `[0.90797067, 0.17650062]`

### Phase 3: detection network and confidence estimator

- Initialization: phase-2 `weights/best.pth.tar`
- Frozen modules: Noiseprint++, backbone, and localization head
- Trained modules: confidence head and detection head
- Training datasets: IMD2020, CASIA 2.0 revised, and CocoGlide
- Training crop: 512 x 512
- Epochs: 100
- Batch size: 4 per GPU
- Validation maximum crop: 1024 x 1024
- Final validation loss: 0.538
- Best `avg_det_bacc`: 0.5981
- Final class IoU: `[0.90785598, 0.08274118]`

## Files

- `weights/best.pth.tar`: best checkpoint selected by `avg_p-F1_smooth`
- `weights/checkpoint.pth.tar`: final epoch-30 resume checkpoint
- `weights/phase3/best.pth.tar`: best phase-3 detection/confidence checkpoint
- `weights/phase3/checkpoint.pth.tar`: final epoch-100 phase-3 resume checkpoint
- `config/trufor_ph2.yaml`: training configuration
- `config/trufor_ph3_gpu2.yaml`: phase-3 training configuration
- `logs/trufor_ph2_gpu1.log`: complete training log
- `logs/trufor_ph3_gpu2.log`: concise phase-3 training log
- `code/`: the locally patched training/device-placement files and launchers
- `SHA256SUMS`: checkpoint integrity hashes

## Loading

These are native TruFor/PyTorch checkpoints, not Transformers checkpoints.
Use them with the TruFor training/inference code. Use
`weights/phase3/best.pth.tar` for complete localization, confidence, and image
detection inference by passing its path through the project's
`TEST.MODEL_FILE` configuration option. Use the top-level
`weights/best.pth.tar` as phase-2 initialization when retraining phase 3.

## How to use

To use or modify this work locally, install Git LFS and run
`git clone https://huggingface.co/benjaik/trufor-ph2`. Use
`weights/phase3/best.pth.tar` for complete inference with the upstream TruFor
code by setting `TEST.MODEL_FILE` to its downloaded path. Use
`weights/best.pth.tar` to initialize another phase-3 run, and use the matching
`checkpoint.pth.tar` file when resuming training. The included configurations,
patched files, and training logs can be copied and adapted for a new dataset or
experiment, subject to the included TruFor and CMX license terms.

## Local compatibility changes

The accompanying files document changes needed on the training server:

- consistent CUDA placement for model, inputs, labels, losses, and resumed
  optimizer state;
- corrected eight malformed CASIA list entries;
- validation crops limited to 1024 pixels to fit an 11 GiB GPU;
- a dedicated CUDA-10.2-compatible Conda environment.

## License and attribution

The upstream TruFor license permits informational and nonprofit use and imposes
additional restrictions. This repository does not relicense the original code
or weights. Review `LICENSE.txt`, `LICENSE_CMX.txt`, and the upstream repository
before use or redistribution.

TruFor paper: *TruFor: Leveraging All-Round Clues for Trustworthy Image Forgery
Detection and Localization*, CVPR 2023.