Text-to-Image
Diffusers
English
stable-diffusion
inference-time-alignment
preference-optimization
pg-map
neurips-2026
Instructions to use sophialan/pg-map-sd15 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use sophialan/pg-map-sd15 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("sophialan/pg-map-sd15", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps
- Draw Things
- DiffusionBee
PG-MAP NeurIPS 2026 — v1.0 custom-pipeline release
Browse files- README.md +86 -0
- pipeline.py +57 -0
README.md
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
language:
|
| 3 |
+
- en
|
| 4 |
+
license: mit
|
| 5 |
+
library_name: diffusers
|
| 6 |
+
tags:
|
| 7 |
+
- text-to-image
|
| 8 |
+
- stable-diffusion
|
| 9 |
+
- inference-time-alignment
|
| 10 |
+
- preference-optimization
|
| 11 |
+
- pg-map
|
| 12 |
+
- neurips-2026
|
| 13 |
+
pipeline_tag: text-to-image
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# PG-MAP for Stable Diffusion 1.5
|
| 17 |
+
|
| 18 |
+
Custom diffusers pipeline for **PG-MAP** (Preference-Guided Adaptive MAP) on SD 1.5. Per-step joint optimization of conditioning $c$ and latent $z_t$ via a trajectory-level Gibbs-MAP / proximal energy objective, optionally guided by a frozen preference reward (PickScore by default).
|
| 19 |
+
|
| 20 |
+
NeurIPS 2026 — see [github.com/sophialanlan/PG-MAP](https://github.com/sophialanlan/PG-MAP) for the paper, full configs, and reproduction scripts.
|
| 21 |
+
|
| 22 |
+
## Install
|
| 23 |
+
|
| 24 |
+
```bash
|
| 25 |
+
pip install pg-map
|
| 26 |
+
# or
|
| 27 |
+
pip install git+https://github.com/sophialanlan/PG-MAP
|
| 28 |
+
```
|
| 29 |
+
|
| 30 |
+
## Usage
|
| 31 |
+
|
| 32 |
+
```python
|
| 33 |
+
from diffusers import DiffusionPipeline
|
| 34 |
+
from pgmap import sd15_defaults, FrozenRewardModel
|
| 35 |
+
import torch
|
| 36 |
+
|
| 37 |
+
pipe = DiffusionPipeline.from_pretrained(
|
| 38 |
+
"runwayml/stable-diffusion-v1-5",
|
| 39 |
+
custom_pipeline="sophialan/pg-map-sd15",
|
| 40 |
+
torch_dtype=torch.float16,
|
| 41 |
+
safety_checker=None,
|
| 42 |
+
).to("cuda")
|
| 43 |
+
|
| 44 |
+
cfg = sd15_defaults() # paper defaults
|
| 45 |
+
reward = FrozenRewardModel("pickscore", device="cuda")
|
| 46 |
+
|
| 47 |
+
image = pipe(
|
| 48 |
+
"a phoenix rising from ashes, vivid orange and red feathers",
|
| 49 |
+
pg_map_config=cfg,
|
| 50 |
+
reward_model=reward,
|
| 51 |
+
).images[0]
|
| 52 |
+
```
|
| 53 |
+
|
| 54 |
+
Passing `pg_map_config=None` falls through to the vanilla `StableDiffusionPipeline`, so the class is a strict superset of the parent.
|
| 55 |
+
|
| 56 |
+
## Method overview
|
| 57 |
+
|
| 58 |
+
Per denoising step $t$, PG-MAP solves the proximal MAP problem:
|
| 59 |
+
|
| 60 |
+
$$\mathcal{J}_t(c, z_t) = -\tfrac{1}{2\beta_{t|s}}\|r_t(c,z_t)\|^2 - \tfrac{1}{2\sigma_c^2}\|c-\mu_t\|^2 - \tfrac{1}{2\sigma_z(t)^2}\|z_t-z_t^{\text{ddim}}\|^2 + \lambda\,Q(\hat x_0(z_t,c), y)$$
|
| 61 |
+
|
| 62 |
+
with $K$ inner gradient-ascent steps and a schedule-adaptive trust region $\sigma_z(t)=\gamma\sqrt{1-\bar\alpha_t}$.
|
| 63 |
+
|
| 64 |
+
## Paper headline (SD 1.5, PartiPrompts $n=1632$, seed 123)
|
| 65 |
+
|
| 66 |
+
| Method | PickScore | HPS | Aesthetic | CLIP |
|
| 67 |
+
|---|---|---|---|---|
|
| 68 |
+
| PG-MAP (default) | **56.8%** | 52.8% | 54.0% | 50.6% |
|
| 69 |
+
| Tuned-CFG + PG-MAP | 53.6% | **66.0%** | **60.2%** | **56.0%** |
|
| 70 |
+
|
| 71 |
+
Win-rate vs. same-seed static baseline.
|
| 72 |
+
|
| 73 |
+
## Citation
|
| 74 |
+
|
| 75 |
+
```bibtex
|
| 76 |
+
@inproceedings{sun2026pgmap,
|
| 77 |
+
title={{PG-MAP}: Joint {MAP} Optimization for Inference-Time Alignment of Diffusion and Flow-Matching Models},
|
| 78 |
+
author={Sun, Ruolan and Polak, Pawel},
|
| 79 |
+
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
|
| 80 |
+
year={2026}
|
| 81 |
+
}
|
| 82 |
+
```
|
| 83 |
+
|
| 84 |
+
## License
|
| 85 |
+
|
| 86 |
+
MIT (see [LICENSE](https://github.com/sophialanlan/PG-MAP/blob/main/LICENSE)). Pretrained weights remain under their original licenses.
|
pipeline.py
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""sophialan/pg-map-sd15 — PG-MAP custom pipeline for Stable Diffusion 1.5.
|
| 2 |
+
|
| 3 |
+
Loaded by the HuggingFace community-pipeline registry::
|
| 4 |
+
|
| 5 |
+
from diffusers import DiffusionPipeline
|
| 6 |
+
pipe = DiffusionPipeline.from_pretrained(
|
| 7 |
+
"runwayml/stable-diffusion-v1-5",
|
| 8 |
+
custom_pipeline="sophialan/pg-map-sd15",
|
| 9 |
+
torch_dtype=torch.float16,
|
| 10 |
+
).to("cuda")
|
| 11 |
+
|
| 12 |
+
This file is a thin re-export shim. The actual pipeline class plus all
|
| 13 |
+
PG-MAP machinery lives in the ``pg-map`` PyPI package; install it first::
|
| 14 |
+
|
| 15 |
+
pip install pg-map
|
| 16 |
+
|
| 17 |
+
Usage after loading::
|
| 18 |
+
|
| 19 |
+
from pgmap import sd15_defaults, FrozenRewardModel
|
| 20 |
+
|
| 21 |
+
cfg = sd15_defaults() # paper defaults
|
| 22 |
+
reward = FrozenRewardModel("pickscore", device="cuda")
|
| 23 |
+
|
| 24 |
+
image = pipe(
|
| 25 |
+
"a phoenix rising from ashes",
|
| 26 |
+
pg_map_config=cfg,
|
| 27 |
+
reward_model=reward,
|
| 28 |
+
).images[0]
|
| 29 |
+
|
| 30 |
+
Passing ``pg_map_config=None`` falls through to the vanilla
|
| 31 |
+
``StableDiffusionPipeline``, so this class is a strict superset of the
|
| 32 |
+
parent.
|
| 33 |
+
|
| 34 |
+
Citation::
|
| 35 |
+
|
| 36 |
+
@inproceedings{sun2026pgmap,
|
| 37 |
+
title={PG-MAP: Joint MAP Optimization for Inference-Time Alignment
|
| 38 |
+
of Diffusion and Flow-Matching Models},
|
| 39 |
+
author={Sun, Ruolan and Polak, Pawel},
|
| 40 |
+
booktitle={NeurIPS},
|
| 41 |
+
year={2026},
|
| 42 |
+
}
|
| 43 |
+
"""
|
| 44 |
+
from __future__ import annotations
|
| 45 |
+
|
| 46 |
+
try:
|
| 47 |
+
from pgmap.pipelines.sd15 import PGMAPStableDiffusionPipeline
|
| 48 |
+
except ImportError as e:
|
| 49 |
+
raise ImportError(
|
| 50 |
+
"Custom pipeline `sophialan/pg-map-sd15` requires the `pg-map` "
|
| 51 |
+
"package.\n"
|
| 52 |
+
" Install from PyPI: pip install pg-map\n"
|
| 53 |
+
" Install from source: pip install git+https://github.com/sophialanlan/PG-MAP\n"
|
| 54 |
+
"Repo: https://github.com/sophialanlan/PG-MAP"
|
| 55 |
+
) from e
|
| 56 |
+
|
| 57 |
+
__all__ = ["PGMAPStableDiffusionPipeline"]
|