Optimal Linear Features for Content Based Image Retrieval and Applications

Since the number of digital images is growing explosively, content based image retrieval becomes an active research area to automatically index and retrieve images based on their semantic features and visual appearance. Content based image retrieval (CBIR) research is largely concentrating on two topics due to their fundamental importance: (1) similarity of images that depends on the feature representation and feature similarity function; (2)machine learning algorithms to enhance retrieval results through adaptively improving classification results and similarity metrics. Color histogram is one of the most commonly used features because it provides important color distribution information of images and is easy to calculate. However, color histogram ignores spatial information which is also important for discriminating for spatial patterns. We propose a new type of features called spetral histogram (SH) features to include spatial information of images by combining local patterns through filters and global features through histograms. Spetral histogram features are obtained by concatenating histograms of image spectral components associated with a bank of filters; it has been shown that they provide a unified representation for modeling textures, faces, and other images. Through experiments, we demonstrate their effectiveness for CBIR using a benchmark dataset. In order to alleviate the sensitivity to scaling, we propose to use "characteristic scale" to obtain intrinsic SH features that are invariant to changes in scale. In order to deal with domain specific images such as "images containing cats", we propose a new shape feature called gradient curve. The gradient curve feature combined with histogram of gradient (HOG) along edge fragment patches is shown to be effective in cat head detection. We develop a new machine learning algorithm called Optimal Factor Analysis (OFA), which is designed to learn low-dimensional representations that optimize discrimination based on the nearest neighbor classifier using Euclidean distances. The method is applied to content-based image categorization and retrieval using SH features. We have achieved significantly better retrieval results on a benchmark dataset than some existing methods. Then we also explore the possibility of improving classification and retrieval result by applying OFA with respect to the metrics derived from cross-correlation of spectral histograms. Considering the large amount of unlabeled data in real world applications, we propose a new semi-supervised learning algorithm named Transductive Optimal Component Analysis (Transductive OCA); it utilizes unlabeled data to learn optimal linear representations by incorporating an additional term that prefers representations with large "margins" when classifying unlabeled data in the nearest classifier sense. We have achieved improvements on face recognition applications using Transductive OCA.