README.md

---
license: mit
language:
  - en
pipeline_tag: image-to-image
library_name: pytorch
tags:
  - e3diff
  - diffusion
  - sar-to-optical
  - image-translation
  - checkpoint
---

> [!WARNING] we do not have a full checkpoint conversion validation, if you encounter pipeline loading failure and unsidered output, please contact me via bili_sakura@zju.edu.cn

# BiliSakura/E3Diff-ckpt

Packaged E3Diff checkpoint for use with `examples/community/e3diff` in `pytorch-image-translation-models`.

## Source repository

- E3Diff (official): [DeepSARRS/E3Diff](https://github.com/DeepSARRS/E3Diff)
- Community implementation used here: [Bili-Sakura/pytorch-image-translation-models](https://github.com/Bili-Sakura/pytorch-image-translation-models)

## Variants

| Variant directory | Notes |
| --- | --- |
| `SEN12 ` | Flat diffusion checkpoint export (`config.json` + `diffusion_pytorch_model.safetensors`) |

## Repository layout

```text
E3Diff-ckpt/
  SEN12 /
    config.json
    diffusion_pytorch_model.safetensors
```

## Usage

Load config and weights from the variant directory directly:

- `config`: `SEN12 /config.json`
- `weights`: `SEN12 /diffusion_pytorch_model.safetensors`

### Inference demo (pipeline)

```python
from PIL import Image

from examples.community.e3diff import E3DiffPipeline

pipe = E3DiffPipeline.from_pretrained(
    "/path/to/E3Diff-ckpt/SEN12 ",
    device="cuda",
)

sar = Image.open("/path/to/sar_input.png").convert("RGB")
out = pipe(source_image=sar, num_inference_steps=50, eta=0.8, output_type="pil")
out.images[0].save("e3diff_output.png")
```

## Citation

```bibtex
@ARTICLE{10767752,
  author={Qin, Jiang and Zou, Bin and Li, Haolin and Zhang, Lamei},
  journal={IEEE Geoscience and Remote Sensing Letters},
  title={Efficient End-to-End Diffusion Model for One-step SAR-to-Optical Translation},
  year={2024},
  pages={1-1},
  doi={10.1109/LGRS.2024.3506566}
}
```