Sparse Representations for Recognition

In recent years, studies have shown that independent/sparse components of local windows of natural images resemble the receptive fields of cells in the early stages of the mammalian visual pathway. However, the role of the independence/sparseness in visual recognition is not well understood. In the first part of this thesis, we argue that the independence/sparseness resolves the curse of dimensionality by reducing the complexity of probability models to the linear order of the dimension. In addition, we show empirically that the complexity reduction does not deteriorate the recognition performance on all the datasets we have used based on proposed independent spectral representation. This study provides the first empirical evidence on the effectiveness of sparse representations for recognition. In the second part of this thesis, we address this question systematically by providing an algorithm for finding sparse representations that are effective for recognition. Although sparse coding has been regarded as an important principle for recognition which has been used effectively to derive filters with desirable properties, there is no effective algorithm to link the sparse coding principle to the recognition performance. By proposing a criterion consisting of weighted combination of recognition performance and sparseness, an optimal sparse linear representation with good recognition performance is achieved by using a Monte-Carlo simulated annealing algorithm. Moreover, we also find an interesting relationship among commonly used linear representations by comparing them based on both sparseness and recognition performance.