hide
Free keywords:
Computer Science, Computer Vision and Pattern Recognition, cs.CV
Abstract:
Most state-of-the-art semi-supervised video object segmentation methods rely
on a pixel-accurate mask of a target object provided for the first frame of a
video. However, obtaining a detailed segmentation mask is expensive and
time-consuming. In this work we explore an alternative way of identifying a
target object, namely by employing language referring expressions. Besides
being a more practical and natural way of pointing out a target object, using
language specifications can help to avoid drift as well as make the system more
robust to complex dynamics and appearance variations. Leveraging recent
advances of language grounding models designed for images, we propose an
approach to extend them to video data, ensuring temporally coherent
predictions. To evaluate our method we augment the popular video object
segmentation benchmarks, DAVIS'16 and DAVIS'17 with language descriptions of
target objects. We show that our approach performs on par with the methods
which have access to a pixel-level mask of the target object on DAVIS'16 and is
competitive to methods using scribbles on the challenging DAVIS'17 dataset.
