license: other
pipeline_tag: image-to-image
FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation
Paper · Project Page · Code
FrameForge adapts the temporal priors of HunyuanVideo 1.5 for three conditional image-generation tasks with one model: subject-driven generation, Canny-to-image generation, and instruction-guided image editing. The condition image is treated as an earlier frame and the output image as a target frame. This repository provides the inference code for the three-expert, one-reasoning-latent configuration.
Usage
For installation and checkpoint setup, please refer to the GitHub repository. Run the commands from the repository root. Replace the example paths with your own reference or condition image, prepared at 512 × 512 pixels. All three tasks generate a single 512 × 512 output image and use 50 denoising steps.
Subject-driven generation
CUDA_VISIBLE_DEVICES=0 MODEL_PATH=./ckpts bash scripts/infer.sh \
--lora_path ./checkpoints/pytorch_lora_weights.safetensors \
--input_image ./inputs/subject.jpg \
--reference_size 512 \
--prompt "the dog playing on a beach" \
--output_path ./outputs/subject.png \
--num_inference_steps 50
Canny-to-image generation
Pass a Canny edge map as the input image. The manuscript uses guidance scale 2.0 for this task:
CUDA_VISIBLE_DEVICES=0 MODEL_PATH=./ckpts bash scripts/infer.sh \
--lora_path ./checkpoints/pytorch_lora_weights.safetensors \
--input_image ./inputs/edges.png \
--reference_size 512 \
--prompt "A red car parked beside a lake at sunset" \
--output_path ./outputs/canny.png \
--num_inference_steps 50 --guidance_scale 2.0
Instruction-guided image editing
Pass the source image and an editing instruction. The default guidance scale is 6.0:
CUDA_VISIBLE_DEVICES=0 MODEL_PATH=./ckpts bash scripts/infer.sh \
--lora_path ./checkpoints/pytorch_lora_weights.safetensors \
--input_image ./inputs/edit.png \
--reference_size 512 \
--prompt "Change the sky to sunset while preserving the rest of the scene" \
--output_path ./outputs/edited.png \
--num_inference_steps 50
Citation
@misc{lin2026frameforge,
title = {FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation},
author = {Yijing Lin and Zhendong Mao and Mengqi Huang and Yuxin Wang and Yongdong Zhang},
year = {2026},
eprint = {2610.36545},
archivePrefix = {arXiv},
primaryClass = {cs.CV},
url = {https://arxiv.org/abs/2610.36545}
}
License
This project is derived from HunyuanVideo 1.5 and retains its Tencent Hunyuan Community License and third-party notices. Please review those terms before redistributing code or weights.