README.md

metadata

title: FrameForge
emoji: 🖼️
colorFrom: purple
colorTo: indigo
sdk: gradio
app_file: app.py
pinned: false
tags:
  - ml-intern
  - frameforge
  - image-editing
  - hunyuanvideo

FrameForge — demo

Unofficial interactive demo for FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation (arXiv 2610.36545), built on the authors' released inference code and official LoRA weights over HunyuanVideo 1.5.

Three modes in one model:

• Instruction-guided editing — "change the sky to sunset" style edits
• Subject-driven generation — place a subject from a photo into a new scene
• Canny-to-image — generate from an edge map (auto-extraction available)

Running it

Built for ZeroGPU (xlarge slices, 96 GB): generation functions are decorated with @spaces.GPU, and torch is pinned to 2.8.0 (the closest ZeroGPU-supported version to the authors' 2.6.0 pin). It also runs unchanged on a persistent GPU (A100 40 GB+ recommended; FF_OFFLOAD=true for smaller GPUs at a large speed cost) — the decorator is a no-op off ZeroGPU.

First run downloads ~40 GB of checkpoints. (byt5-small/Glyph-SDXL-v2 from the authors' README are skipped — the released config sets disable_by5=True, so they are never loaded.) On Spaces, attach persistent storage and set FF_CKPT_DIR=/data/ckpts so this happens once.

Quota note: a 50-step generation takes a few minutes of GPU time and xlarge consumes 2× daily quota — free accounts (5 min/day) get roughly one image per day, PRO accounts (40 min/day) get several.

Required secret: HF_TOKEN — the pipeline pulls the gated FLUX.1-Redux-dev vision encoder, so the token's account must have accepted its license.

Useful env vars: FF_CKPT_DIR (checkpoint root), FF_LORA_PATH, FF_REPO_DIR (code cache), FF_OFFLOAD (true/false).

Local run

pip install -r requirements.txt
python app.py

Notes

• Outputs are 512×512; ~50 denoising steps on a video-model backbone, so expect a couple of minutes per image.
• License: Tencent Hunyuan Community License (see the base model and the FrameForge repo) — review before redistribution or commercial use.
• If you use this in research, cite the paper: https://arxiv.org/abs/2610.36545