File size: 3,356 Bytes
44b8073 2f792b9 44b8073 2f792b9 c0196d7 4f224ad c0196d7 2f792b9 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | ---
license: cc-by-nc-sa-4.0
library_name: pytorch
tags:
- image-fusion
- infrared-visible
- perceptual-quality
- pairwise-preference
- bradley-terry
pipeline_tag: image-classification
---
# LPIFM — Learned Perceptual Image Fusion Measure
Pairwise perceptual preference model for **infrared–visible image fusion (IVIF)** ranking.
Given an IR source, a VI source, and two fused candidates, LPIFM predicts **A better**, **B better**, or **Tie**. Pairwise decisions can be aggregated with tie-aware Bradley–Terry (T-BT) to rank a method pool.
Code and examples: [github.com/HaoranLiu507/LPIFM](https://github.com/HaoranLiu507/LPIFM)
## Files
| File | Role |
| --- | --- |
| `lpifm_vifb_baseline_v1.pt` | Main VIFB-trained public checkpoint |
| `inference_config.yaml` | Decode defaults (`t`, `T_cal`, image size) |
EVAFusion fine-tuned weights are **not** hosted here; see the Zenodo companion archive when published.
**Note on checkpoint size.**
`lpifm_vifb_baseline_v1.pt` is about **2 GB** because it is a full training checkpoint: three weight copies (`swa` / `ema` / `model`, about 0.4 GB each) plus the optimizer (about 0.8 GB). To **use LPIFM** only, keep `swa_state_dict` (+ `config`); about 0.4 GB is enough. Inference already loads SWA by default.
## Architecture
- Backbone: **ConvNeXt-V2** (`convnextv2_base.fcmae_ft_in22k_in1k_384` via `timm`)
- Input size: **384 × 384**
- Task: source-conditioned pairwise preference scoring (ternary A / B / Tie)
## Decode / inference defaults
| Parameter | Value |
| --- | --- |
| Tie threshold `t` | `0.3` |
| Calibration temperature `T_cal` | `1.0` |
| Image size | `384` |
```text
d_cal = d / T_cal
d_cal > 0.3 → A better (0)
d_cal < -0.3 → B better (1)
otherwise → Tie (2)
```
## Quick start
```bash
# from the GitHub repository
python scripts/download_assets.py --source hf
python predict.py \
--config configs/release_inference.yaml \
--dataset_root Dataset/VIFB \
--ckpt checkpoints/lpifm_vifb_baseline_v1.pt \
--image_name carLight.jpg \
--a_dir U2Fusion \
--b_dir SeAFusion
```
Or download this file directly:
```bash
huggingface-cli download FengShaner/LPIFM lpifm_vifb_baseline_v1.pt \
--local-dir checkpoints
```
## Intended use
- Pairwise **perceptual preference** for **IVIF method ranking**
- Research / offline evaluation under the LPIFM protocol
- Not a general-purpose IQA model for arbitrary natural-image aesthetics
## Limitations
- Trained and validated under a specific preference-collection and ranking protocol
- Protocol mismatch (new dataset, different rater instructions, different method pools) may reduce agreement; fine-tuning may be required
- Non-commercial weights license (see below)
## Licenses
| Artifact | License |
| --- | --- |
| Model weights on this Hub repo | [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) |
| Source code on GitHub | [AGPL-3.0](https://github.com/HaoranLiu507/LPIFM/blob/main/LICENSE) |
Weights do **not** inherit AGPL; code does **not** inherit CC BY-NC-SA.
## Links
- GitHub: https://github.com/HaoranLiu507/LPIFM
- Hugging Face: https://huggingface.co/FengShaner/LPIFM
- Zenodo archival DOI: to be added after deposit publication
## Citation
See [`CITATION.cff`](https://github.com/HaoranLiu507/LPIFM/blob/main/CITATION.cff) in the GitHub repository.
|