Item

Abstract

Redundancy reduction has been proposed as a principle underlying the self-organization of neural representations at the early stages of sensory processing [1]. In particular, principal component analysis (PCA), symmetric whitening (SWH) and independent component analysis (ICA) have been studied as parsimonious redundancy reduction models. When applied to data sets of natural image patches second-order decorrelation methods such as PCA and SWH do not yield localized, oriented, and bandpass filter shapes. These striking properties of V1 simple cell receptive fields, however, can be derived with ICA because of its additional minimization of higher-order correlations. While this finding is intriguing, the structure of the higher-order correlations encountered in ICA is not well understood and their use for sensory coding remains elusive.
Previous studies [2-4] have tried to quantify the difference in coding efficiency between the orientation selective ICA filters and those derived with second-order decorrelation methods. Due to the different methods used, these studies yielded differing results for the coding gain of ICA. In a comprehensive study we included all the previous approaches by measuring the expected log-likelihood, the multi-information, as well as rate-distortion curves for both gray-level and color images. Without exception, we find that the advantage of ICA is very small. We further corroborate and explain this finding by showing that a spherical symmetric distribution can fit the data even better than the ICA model. For this model all filter shapes are equally well suited since the distribution is invariant under arbitrary orthogonal transforms. In conclusion, more sophisticated models are necessary to explain V1 receptive field properties in terms of optimal coding principles.