File size: 3,896 Bytes
0997ec9 39f95db 0997ec9 39f95db 0997ec9 39f95db 0997ec9 39f95db 0997ec9 39f95db 0997ec9 39f95db 0997ec9 39f95db 0997ec9 a54d992 39f95db a54d992 0997ec9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | ---
license: other
library_name: pytorch
tags:
- image-forensics
- image-manipulation-detection
- image-segmentation
- trufor
- pytorch
---
# TruFor Phase-2 and Phase-3 Checkpoints
This repository stores phase-2 localization and phase-3 detection/confidence
checkpoints trained with the official
[TruFor](https://github.com/grip-unina/TruFor) PyTorch implementation. TruFor
detects and localizes manipulated regions in images using RGB and Noiseprint++
features.
## Training summary
- Architecture: TruFor localization network (`detconfcmx`, SegFormer-B2 backbone)
- Training datasets: IMD2020, CASIA 2.0 revised, and CocoGlide
- Training crop: 512 x 512
- Epochs: 30
- Optimizer: SGD
- Initial learning rate: 0.005
- Batch size: 1 per GPU
- Hardware: NVIDIA GeForce RTX 2080 Ti (11 GiB)
- Validation maximum crop: 1024 x 1024
Final recorded validation results:
- Loss: 0.657
- Best `avg_p-F1_smooth`: 0.5505
- Class IoU: `[0.90797067, 0.17650062]`
### Phase 3: detection network and confidence estimator
- Initialization: phase-2 `weights/best.pth.tar`
- Frozen modules: Noiseprint++, backbone, and localization head
- Trained modules: confidence head and detection head
- Training datasets: IMD2020, CASIA 2.0 revised, and CocoGlide
- Training crop: 512 x 512
- Epochs: 100
- Batch size: 4 per GPU
- Validation maximum crop: 1024 x 1024
- Final validation loss: 0.538
- Best `avg_det_bacc`: 0.5981
- Final class IoU: `[0.90785598, 0.08274118]`
## Files
- `weights/best.pth.tar`: best checkpoint selected by `avg_p-F1_smooth`
- `weights/checkpoint.pth.tar`: final epoch-30 resume checkpoint
- `weights/phase3/best.pth.tar`: best phase-3 detection/confidence checkpoint
- `weights/phase3/checkpoint.pth.tar`: final epoch-100 phase-3 resume checkpoint
- `config/trufor_ph2.yaml`: training configuration
- `config/trufor_ph3_gpu2.yaml`: phase-3 training configuration
- `logs/trufor_ph2_gpu1.log`: complete training log
- `logs/trufor_ph3_gpu2.log`: concise phase-3 training log
- `code/`: the locally patched training/device-placement files and launchers
- `SHA256SUMS`: checkpoint integrity hashes
## Loading
These are native TruFor/PyTorch checkpoints, not Transformers checkpoints.
Use them with the TruFor training/inference code. Use
`weights/phase3/best.pth.tar` for complete localization, confidence, and image
detection inference by passing its path through the project's
`TEST.MODEL_FILE` configuration option. Use the top-level
`weights/best.pth.tar` as phase-2 initialization when retraining phase 3.
## How to use
To use or modify this work locally, install Git LFS and run
`git clone https://huggingface.co/benjaik/trufor-ph2`. Use
`weights/phase3/best.pth.tar` for complete inference with the upstream TruFor
code by setting `TEST.MODEL_FILE` to its downloaded path. Use
`weights/best.pth.tar` to initialize another phase-3 run, and use the matching
`checkpoint.pth.tar` file when resuming training. The included configurations,
patched files, and training logs can be copied and adapted for a new dataset or
experiment, subject to the included TruFor and CMX license terms.
## Local compatibility changes
The accompanying files document changes needed on the training server:
- consistent CUDA placement for model, inputs, labels, losses, and resumed
optimizer state;
- corrected eight malformed CASIA list entries;
- validation crops limited to 1024 pixels to fit an 11 GiB GPU;
- a dedicated CUDA-10.2-compatible Conda environment.
## License and attribution
The upstream TruFor license permits informational and nonprofit use and imposes
additional restrictions. This repository does not relicense the original code
or weights. Review `LICENSE.txt`, `LICENSE_CMX.txt`, and the upstream repository
before use or redistribution.
TruFor paper: *TruFor: Leveraging All-Round Clues for Trustworthy Image Forgery
Detection and Localization*, CVPR 2023.
|