| | --- |
| | library_name: diffusers |
| | pipeline_tag: image-to-video |
| | --- |
| | |
| | # Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance |
| |
|
| | Kiwi-Edit is a versatile video editing framework built on an MLLM encoder and a video Diffusion Transformer (DiT). It supports both instruction-based video editing and reference-guided editing (using a reference image and instruction). |
| |
|
| | - **Paper:** [Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance](https://huggingface.co/papers/2603.02175) |
| | - **Project Page:** [https://showlab.github.io/Kiwi-Edit/](https://showlab.github.io/Kiwi-Edit/) |
| | - **Repository:** [https://github.com/showlab/Kiwi-Edit](https://github.com/showlab/Kiwi-Edit) |
| |
|
| | ## Model Description |
| |
|
| | Kiwi-Edit introduces a unified editing architecture that synergizes learnable queries and latent visual features for reference semantic guidance. It addresses the challenge of precise visual control in instruction-based editing by allowing users to provide a reference image to guide the transformation. The framework achieves significant performance improvements in instruction following and reference fidelity through a scalable data generation pipeline and a multi-stage training curriculum. |
| |
|
| | ## Usage |
| |
|
| | This model is compatible with the `diffusers` library. To run inference, follow the installation instructions in the [official repository](https://github.com/showlab/Kiwi-Edit). |
| |
|
| | ### Quick Test with Diffusers |
| |
|
| | You can run a quick test on a demo video using the following command provided in the repository: |
| |
|
| | ```bash |
| | python diffusers_demo.py \ |
| | --video_path ./demo_data/video/source/0005e4ad9f49814db1d3f2296b911abf.mp4 \ |
| | --prompt "Remove the monkey." \ |
| | --save_path output.mp4 \ |
| | --model_path linyq/kiwi-edit-5b-instruct-only-diffusers |
| | ``` |
| |
|
| | ## Citation |
| |
|
| | If you find this work useful, please cite: |
| |
|
| | ```bibtex |
| | @misc{kiwiedit, |
| | title={Kiwi-Edit: Versatile Video Editing via Instruction and Reference Guidance}, |
| | author={Yiqi Lin and Guoqiang Liang and Ziyun Zeng and Zechen Bai and Yanzhe Chen and Mike Zheng Shou}, |
| | year={2026}, |
| | eprint={2603.02175}, |
| | archivePrefix={arXiv}, |
| | primaryClass={cs.CV}, |
| | url={https://arxiv.org/abs/2603.02175}, |
| | } |
| | ``` |
