hide
Free keywords:
Computer Science, Computer Vision and Pattern Recognition, cs.CV
Abstract:
We present GazeDirector, a new approach for eye gaze redirection that uses
model-fitting. Our method first tracks the eyes by fitting a multi-part eye
region model to video frames using analysis-by-synthesis, thereby recovering
eye region shape, texture, pose, and gaze simultaneously. It then redirects
gaze by 1) warping the eyelids from the original image using a model-derived
flow field, and 2) rendering and compositing synthesized 3D eyeballs onto the
output image in a photorealistic manner. GazeDirector allows us to change where
people are looking without person-specific training data, and with full
articulation, i.e. we can precisely specify new gaze directions in 3D.
Quantitatively, we evaluate both model-fitting and gaze synthesis, with
experiments for gaze estimation and redirection on the Columbia gaze dataset.
Qualitatively, we compare GazeDirector against recent work on gaze redirection,
showing better results especially for large redirection angles. Finally, we
demonstrate gaze redirection on YouTube videos by introducing new 3D gaze
targets and by manipulating visual behavior.
