Zhu, Lianghan, Bao, Yanqi, Huo, Jing, Wu, Jing  ORCID: https://orcid.org/0000-0001-5123-9861, Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680, Li, Wenbin and Gao, Yang
2027.
Training-free video editing via optical flow-enhanced score distillation.
Pattern Recognition
183
(C)
, 115077.
10.1016/j.patcog.2026.115077
|
Preview |
PDF
- Accepted Post-Print Version
Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (62MB) | Preview |
Abstract
The rapid advancement in visual generation, particularly the emergence of pre-trained text-to-image and text-to-video models, has catalyzed growing interest in training-free video editing research. Mirroring training-free image editing techniques, current approaches preserve original video information through video input inversion and manipulation of intermediate features and attention during the inference process to achieve content editing. Although they have demonstrated promising results, the lossy nature of the inversion process poses significant challenges in maintaining unedited regions of the video. Furthermore, approaches based on feature and attention manipulation during inference can lead to unintended over-editing and face challenges in both local temporal continuity and global content consistency. To address these challenges, this study proposes a score distillation paradigm based on pre-trained text-to-video models, where the original video is iteratively optimized through multiple steps guided by editing gradients provided by score distillation to ultimately obtain the target video. The iterative optimization starting from the original video, combined with a content preservation loss, ensures the maintenance of unedited regions in the original video and suppresses over-editing. To further guarantee video content consistency and temporal continuity, we additionally introduce a global consistency auxiliary loss and optical flow prediction-based local editing gradient smoothing. Experiments demonstrate that these strategies effectively address the aforementioned challenges, achieving comparable or superior performance across multiple dimensions, including preservation of unedited regions, local temporal continuity, and global content consistency of editing results, compared to that of state-of-the-art methods.
| Item Type: | Article |
|---|---|
| Date Type: | Publication |
| Status: | Published |
| Schools: | Schools > Computational & Mathematical Sciences Schools > Computer Science & Informatics |
| Additional Information: | RRS applied |
| Publisher: | Elsevier |
| ISSN: | 0031-3203 |
| Date of First Compliant Deposit: | 27 October 2026 |
| Date of Acceptance: | 30 September 2026 |
| Last Modified: | 03 Nov 2026 11:00 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/190391 |
Actions (repository staff only)
 |
Edit Item |
Altmetric
Altmetric