hide
Free keywords:
Computer Science, Human-Computer Interaction, cs.HC
Abstract:
Users' visual attention is highly fragmented during mobile interactions but
the erratic nature of these attention shifts currently limits attentive user
interfaces to adapt after the fact, i.e. after shifts have already happened,
thereby severely limiting the adaptation capabilities and user experience. To
address these limitations, we study attention forecasting -- the challenging
task of predicting whether users' overt visual attention (gaze) will shift
between a mobile device and environment in the near future or how long users'
attention will stay in a given location. To facilitate the development and
evaluation of methods for attention forecasting, we present a novel long-term
dataset of everyday mobile phone interactions, continuously recorded from 20
participants engaged in common activities on a university campus over 4.5 hours
each (more than 90 hours in total). As a first step towards a fully-fledged
attention forecasting interface, we further propose a proof-of-concept method
that uses device-integrated sensors and body-worn cameras to encode rich
information on device usage and users' visual scene. We demonstrate the
feasibility of forecasting bidirectional attention shifts between the device
and the environment as well as for predicting the first and total attention
span on the device and environment using our method. We further study the
impact of different sensors and feature sets on performance and discuss the
significant potential but also remaining challenges of forecasting user
attention during mobile interactions.
