| --- |
| license: mit |
| pipeline_tag: text-to-image |
| library_name: pytorch |
| tags: |
| - hdr |
| - hdr-generation |
| - diffusion |
| - guidance |
| - training-free |
| - flux |
| - distribution-shaping |
| - luminance |
| base_model: |
| - black-forest-labs/FLUX.1-dev |
| base_model_relation: adapter |
| --- |
| |
| # LumaGuide: text-to-histogram regressor and test prompts |
|
|
| Assets for **LumaGuide: Distribution Shaping for Training-Free HDR Generation in Diffusion Models** (Bowen Chen, Shreshth Saini, Balu Adsumilli, Alan C. Bovik). |
|
|
| - Paper: https://arxiv.org/abs/2607.26237 |
| - Code: https://github.com/shreshthsaini/LumaGuide |
| - Project page: https://shreshthsaini.github.io/LumaGuide/ |
|
|
| LumaGuide is training-free. It steers a pretrained diffusion model at sampling time so that a feature distribution of the output matches a target, using differentiable energy-based guidance. For HDR generation the feature is the PQ-space luminance histogram, matched with a Wasserstein-1 loss back-propagated through the VAE decoder of Flux.1-dev. The diffusion transformer is never trained or modified. |
|
|
| ## What is in this repo |
|
|
| | File | Purpose | |
| | --- | --- | |
| | `regressor.pt` | Caption-to-histogram regressor: a 3-layer MLP (768 to 256 to 256 to 32, GELU) on the pooled `openai/clip-vit-large-patch14` text embedding, softmax output over 32 PQ-luminance bins. 270,880 parameters, 1.1 MB. This is the default target source for `--target regressor`. | |
| | `test_prompts.csv` | The 100 HDR test prompts used in the paper, with generation width and height. | |
|
|
| The regressor only provides the target histogram. The guidance itself, the PQ luminance utilities, and the Flux sampler hooks live in the GitHub repo. |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/shreshthsaini/LumaGuide && cd LumaGuide |
| pip install -r requirements.txt |
| huggingface-cli download shreshthsaini/LumaGuide regressor.pt --local-dir . |
| python lumaguide.py --prompt "A skier at the top of a glacier as the sun rises" \ |
| --target regressor --regressor_ckpt regressor.pt --out out.hdr |
| ``` |
|
|
| Or load the checkpoint yourself: |
|
|
| ```python |
| import torch |
| from huggingface_hub import hf_hub_download |
| state = torch.load(hf_hub_download("shreshthsaini/LumaGuide", "regressor.pt"), map_location="cpu", weights_only=False) |
| state["config"] # {'in_dim': 768, 'hidden': 256, 'n_bins': 32} |
| state["model_state"] # weights for HistRegressor in lumaguide.py |
| ``` |
|
|
| Flux.1-dev is downloaded separately from Hugging Face and is gated; accept its license first. It needs about 30 GB of VRAM at 512x512 with LumaGuide's VAE backward pass. |
|
|
| ## License |
|
|
| The regressor and prompts are released under MIT. Flux.1-dev is governed by the Black Forest Labs non-commercial license. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{chen2026lumaguide, |
| title = {LumaGuide: Distribution Shaping for Training-Free HDR Generation in Diffusion Models}, |
| author = {Chen, Bowen and Saini, Shreshth and Adsumilli, Balu and Bovik, Alan C.}, |
| journal = {arXiv preprint arXiv:2607.26237}, |
| year = {2026} |
| } |
| ``` |
|
|
