A variational model for gradient-based video editing


Abstract: In this work we present a gradient-based variational model for video editing, addressing the problem of propagating gradient-domain information along the optical flow of the video. The resulting propagation is temporally consistent and blends seamlessly with its spatial surroundings. In addition, the presented model is able to cope with additive illumination changes and handles occlusions/dis-occlusions. The problem of propagation along the optical flow arises in different video editing applications. In this work we consider the application where a user edits a frame by modifying the texture of an object's surface and wishes to propagate this editing throughout the video.


Full paper: "A Variational Model for Gradient-Based Video Editing", IJCV, 2012.

Supplementary Materials:

The supplementary material document to the sumbitted paper can be downloaded from here

A MATLAB code with data to test it can be downloaded from here, or one could download only the MATLAB code here and the different testing data seperately: magazine seq, cloth tilt sequence and hand sequence

Note 1: The animations suffer from some distortions due to downsampling and compression. Please download the "avi's" or "png's" to get the full un-compressed results.

Note 2: The animations will run just once, click on each figure to repeat. One could always download the original "avi's" or "png's".

One lid setting experiments
Sequence name Input sequence Editing domain Result Download
Coca-cola sequence a small gif animation of the original coca-cola sequence, the edit domain and the result avi png
Magazine sequence a small gif animation of the magazine experiment, the edit domain and the result avi png
Cloth wave a small gif animation of the cloth wave sequence, the edit domain and the result avi png
Can and Box sequence a small gif animation of the can and box experiment, the edit domain and the 1-lid result avi png
Two lid setting experiments
Sequence name Input sequence Editing domain Result Download
Can and Box sequence a small gif animation of the can and box experiment, the edit domain ant the 2-lid result avi png
Screen sequence
(The files to download
include results with
different optical flow algos)		 a small gif animation of the screen experiment, the edit domain and the 2-lid result avi png
Pen holder sequence a small gif animation of the pen holder experiment, the edit domain and the 2-lid result avi png
Double occlusion
Sequence name Input sequence Editing domain Result (not handled) Result (corrected) Download
Hand (double occlusion) a small gif animation of the hand experiment, the edit domain, a result without handling the double occlusion and the result obtained with occlusion handling avi (not handled)

avi (corrected)		 png (not handled)

png (corrected)
A zoom-in case (we edit the last frame and propagate backwards)
Sequence name Input sequence Editing domain Result Download
Cliff (a zoom in sequence) a small gif animation of the cliff experiment, the edit domain and the result avi png
Zoom and tilt
Sequence name Input sequence Editing domain Result (1-lid bilinear) Result (1-lid bicubic) Result (2-lid bilinear) Download
Cloth tilt a small gif animation of a zoom and tilt experiment, the edit domain, the 1-lid result using bilinear interpolation, the 1-lid result using bicubic interpolation and the 2-lid result with bilinear interpolation avi (1-lid bilinear) png (1-lid bilinear) avi (1-lid bicubic) png (1-lid bicubic) avi (2-lid bilinear) png (2-lid bilinear)

To download all the videos please click here.

To download all the image frames please click here.