FastVMT: Eliminating Redundancy in Video Motion Transfer
Abstract
FastVMT accelerates video motion transfer by addressing computational redundancies in Diffusion Transformer architecture through localized attention masking and gradient reuse optimization.
Video motion transfer aims to synthesize videos by generating visual content according to a text prompt while transferring the motion pattern observed in a reference video. Recent methods predominantly use the Diffusion Transformer (DiT) architecture. To achieve satisfactory runtime, several methods attempt to accelerate the computations in the DiT, but fail to address structural sources of inefficiency. In this work, we identify and remove two types of computational redundancy in earlier work: motion redundancy arises because the generic DiT architecture does not reflect the fact that frame-to-frame motion is small and smooth; gradient redundancy occurs if one ignores that gradients change slowly along the diffusion trajectory. To mitigate motion redundancy, we mask the corresponding attention layers to a local neighborhood such that interaction weights are not computed unnecessarily distant image regions. To exploit gradient redundancy, we design an optimization scheme that reuses gradients from previous diffusion steps and skips unwarranted gradient computations. On average, FastVMT achieves a 3.43x speedup without degrading the visual fidelity or the temporal consistency of the generated videos.
Community
FastVMT speeds up video motion transfer by masking local attention and reusing gradients to remove motion and gradient redundancy, achieving 3.43x speedup without quality loss.
arXivLens breakdown of this paper ๐ https://arxivlens.com/PaperView/Details/fastvmt-eliminating-redundancy-in-video-motion-transfer-1408-0dfb29fc
- Executive Summary
- Detailed Breakdown
- Practical Applications
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- MotionAdapter: Video Motion Transfer via Content-Aware Attention Customization (2026)
- SMRABooth: Subject and Motion Representation Alignment for Customized Video Generation (2025)
- V-Warper: Appearance-Consistent Video Diffusion Personalization via Value Warping (2025)
- HiStream: Efficient High-Resolution Video Generation via Redundancy-Eliminated Streaming (2025)
- IC-Effect: Precise and Efficient Video Effects Editing via In-Context Learning (2025)
- Latent Temporal Discrepancy as Motion Prior: A Loss-Weighting Strategy for Dynamic Fidelity in T2V (2026)
- Memory-V2V: Augmenting Video-to-Video Diffusion Models with Memory (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper