Item

Abstract

When a person looks at an object while exploring it with their hand, vision and touch
both provide useful information for estimating the properties of the object. Here, we
investigated the fusion of visual and haptic information and its limits. We propose that a
general principle, which minimizes variance in the final estimate, determines the degree
to which vision or haptics determines the integrated, visual-haptic percept. This principle
is realized by using maximum-likelihood estimation (MLE) to combine the inputs. To
investigate cue combination quantitatively, we first measured the variances associated
with visual and haptic estimation of height. Those measurements were then used to construct
the MLE integrator. The model and humans behaved very similarly in a visual-haptic
height discrimination task. Thus, the nervous system seems to combine visual and
haptic height information in a fashion quite similar to MLE integration. In a second study
we used an oddity task to investigate the break down of visual-haptic fusion. Three horizontal
bars were presented sequentially. Two of them were identical and had equal visual
and haptic heights (standard stimulus); the third had a visual and/or haptic height differing
from the standard (odd stimulus). Subjects indicated which of the three intervals contained
the odd stimulus. If subjects relied on the fused visual-haptic estimate, discrimination
should be most difficult when the weighted average of the visual and haptic heights
corresponds to the standards height and easiest if both heights of the odd stimulus are
either bigger or smaller then the standards height. In contrast, if subjects used the visual
or haptic information independently without fusing them, discrimination would occur
whenever either the visual or the haptic height in the odd stimulus differed noticeably
from the standards height. We found that discrimination was indeed most difficult when
the weighted averages were the same and easiest when they differed. Thus, the fused
visual-haptic percept is used for discriminating between the stimuli. However, if the conflict
between the visual and haptic stimuli became too large, this difference in discrimination
performance was not observed anymore. In other words, visual-haptic fusion breaks
with large conflicts. Remarkably, we observed metameric behavior in some condition.
That is, discrimination would be better if subjects simply shut their eyes or removed their
hand from the bar instead of using the fused visual-haptic information.