README.md

metadata

title: Foveated Diffusion Demo
emoji: 👁️
colorFrom: gray
colorTo: blue
sdk: gradio
sdk_version: 6.14.0
app_file: app.py
pinned: false
license: apache-2.0
suggested_hardware: l40sx1
hf_oauth: false
startup_duration_timeout: 5h
tags:
  - lora
  - diffusion
  - foveated-rendering
  - text-to-image
  - flux
github: https://github.com/bchao1/foveated_diffusion

Foveated Diffusion — live demo

Live demo of Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generation. Place a circular foveal region you want rendered at high resolution, pick a LoRA adapter, and run a 50-step inference pass through the foveated FLUX.2 pipeline.

• Project page: https://bchao1.github.io/foveated-diffusion/
• Paper: https://arxiv.org/abs/2603.23491
• Code: https://github.com/bchao1/foveated_diffusion
• Model weights: https://huggingface.co/bchao1/foveated_diffusion

How to use

1. Type a prompt and pick a seed.
2. Choose a LoRA from the dropdown:
   • no_fov — finetuned baseline, no foveation conditioning (always full-HR).
   • random — foveation conditioning at random gaze locations.
   • saliency — saliency-driven foveation masks.
3. Set the foveal circle: center cx, cy ∈ [-0.5, 0.5] (0 = image center) and radius r (relative to half the image diagonal). With no_fov the controls are disabled — the whole image is rendered HR.
4. The Tokenization mask preview shows exactly which 32×32-px blocks will be rendered at HR (white) vs LR (gray) before you commit.
5. Click Generate. The denoising progress bar reports steps k / 50.

Fixed pipeline settings

Setting	Value
Resolution	1024 × 1024
Steps	50
CFG	4.0
Decode mode	merge
Prediction type	clean
LR downsample factor	2
Soft foveation blend	on

Hardware

FLUX.2-klein-base-4B + LoRA in bfloat16 needs ~30 GB of VRAM. We recommend running this Space on an L40S (l40sx1) or larger GPU tier. The CPU-Basic free tier cannot host the model.

Citation

@misc{chao2026foveateddiffusion,
      title={Foveated Diffusion: Efficient Spatially Adaptive Image and Video Generation},
      author={Brian Chao and Lior Yariv and Howard Xiao and Gordon Wetzstein},
      year={2026},
      eprint={2603.23491},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2603.23491},
}