Item

Abstract

The appearance of objects in an image can change dramatically depending on their pose, distance, and illumination. Learning representations that are invariant against such appearance changes can be viewed as an important preprocessing step which removes distracting variance from a data set, so that downstream classifiers or regression estimators perform better. Complex cells in primary visual cortex are commonly seen as building blocks for such invariant image representations (e.g. Riesenhuber Poggio 2000). While complex cells, like simple cells, respond to edges of particular orientation they are less sensitive to the precise location of the edge. A variety of neural algorithms have been proposed that aim at explaining the response properties of complex cells as components of an invariant representation that is optimized for the spatio-temporal statistics of the visual input. For certain classes of transformations (e.g. translations, scalings, and rotations), it is possible to analytically derive features that are invariant under these transformations, and the design of such invariant features has been studied extensively in computer vision. The range of naturally occurring transformations, however, is much more variable and not precisely known. Thus, an analytical design of invariant features does not seem feasible. Instead one can seek to find features that may not be perfectly invariant anymore but which on average change as slowly as possible under the transformations occurring in the data (Földiák 1991). The best known instantiation of this approach is slow feature analysis (SFA) which has been proposed to underlie the formation of complex cell receptive fields (Berkes Wiskott 2005). From a machine learning perspective, SFA can be seen as a special case of oriented principal component analysis that greedily searches for filters that maximize the signal-to-noise ratio if the variations generated by the transformational changes are considered noise. For the learning of complex cells the algorithm has been applied in the quadratic feature space. Here we present a new algorithm called slow subspace analysis (SSA). SSA combines the slowness objective of SFA with the energy model known from steerable filter theory such that it yields perfectly invariant steerable filters in the ideal analytically tractable cases. There are two important differences between SFA and SSA: First, while SSA uses the same slowness criterion as SFA for each individual feature, it replaces the greedy search strategy by optimizing all filters simultaneously for the best average slowness, and second, the optimization in SSA is done only over the (n2 + n)/2 dimensional parameter space of orthogonal transforms on the original n-dimensional signal space while for complex cell learning with SFA the optimization is carried out over the entire quadratic feature space for which the number of parameters is much larger, i.e. (n4+2n3−n2−2n)/8. These differences make SSA an interesting alternative to SFA. In particular, the theoretical grounding of SSA in steerable filter theory is attractive as it allows one to carry out meaningful model comparisons between different algorithms. Accordingly, we show that our new algorithm exhibits larger slowness than SFA for various important examples such as translations, rotations and scalings as well as natural movies.