Multiple kernel nonnegative matrix factorization

2014 IEEE International Conference on Image Processing (ICIP), 2014

As a considerable technique in image processing and computer vision, Nonnegative Matrix Factorization (NMF) generates its bases by iteratively multiplicative update with two initial random nonnegative matrices W and H, that leads to the randomness of the bases selection. For this reason, the potentials of NMF algorithms are not completely exploited. To address this issue, we present a novel framework which uses the feature selection techniques to evaluate and rank the bases of the NMF algorithms to enhance the NMF algorithms. We adopted the well known Fisher criterion and Least Reconstruction Error criterion, which is proposed by us, as two instances to show how that works successfully under our framework. Moreover, in order to avoid the hard combinatorial optimization issue in ranking procedure, a de-correlation constraint can be optionally imposed to the NMF algorithms for giving a better approximation to the global optimum of the NMF projections. We evaluate our works in face recognition, object recognition and image reconstruction on ORL and ETH-80 databases and the results demonstrate the enhancement of the state-of-the-art NMF under our framework.

CVPR 2011, 2011

Non-negative matrix factorization (NMF) has previously been shown to be a useful decomposition tool for multivariate data. Non-negative bases allow strictly additive combinations which have been shown to be part-based as well as relatively sparse. We pursue a discriminative decomposition by coupling NMF objective with a maximum margin classifier, specifically a support vector machine (SVM). Conversely, we propose an NMF based regularizer for SVM. We formulate the joint update equations and propose a new method which identifies the decomposition as well as the classification parameters. We present classification results on synthetic as well as real datasets.

2012 IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), 2012

Non-negative factorization (NMF) has been a popular machine learning method for analyzing microarray data. Kernel approaches can capture more non-linear discriminative features than linear ones. In this paper, we propose a novel kernel NMF (KNMF) approach for feature extraction and classification of microarray data. Our approach is also generalized to kernel high-order NMF (HONMF). Extensive experiments on eight microarray datasets show that our approach generally outperforms the traditional NMF and existing KNMFs. Preliminary experiment on a high-order microarray data shows that our KHONMF is a promising approach given a suitable kernel function.

Pattern Recognition Letters, 2011

Nonnegative matrix factorization (NMF) is an unsupervised learning method for low-rank approximation of nonnegative data, where the target matrix is approximated by a product of two nonnegative factor matrices. Two important ingredients are missing in the standard NMF methods: (1) discriminant analysis with label information; (2) geometric structure (manifold) in the data. Most of the existing variants of NMF incorporate one of these ingredients into the factorization. In this paper, we present a variation of NMF which is equipped with both these ingredients, such that the data manifold is respected and label information is incorporated into the NMF. To this end, we regularize NMF by intra-class and inter-class knearest neighbor (k-NN) graphs, leading to NMF-kNN, where we minimize the approximation error while contracting intra-class neighborhoods and expanding inter-class neighborhoods in the decomposition. We develop simple multiplicative updates for NMF-kNN and present monotonic convergence results. Experiments on several benchmark face and document datasets confirm the useful behavior of our proposed method in the task of feature extraction.

In this paper, a novel method for facial representation called Spatially Confined Non-Negative Matrix Factorization (SFNMF) is presented. SFNMF aims to extract more spatially confined, parts-based representation from the NMF based representation by merely removing non-prominent region, and focalize on the salient feature. SFNMF derived a significant set of basis which allows a non-subtractive representation of images and these bases manifest localized features. Experimental results are presented to compare SFNMF with NMF and Local NMF. Advantageous of SFNMF is demonstrated when SFNMF achieves highest verification rate among the other.

IEEE Transactions on Information Forensics and Security, 2007

The methods introduced so far regarding Discriminant Non-negative Matrix Factorization (DNMF) do not guarantee convergence to a stationary limit point. In order to remedy this limitation, a novel DNMF method is presented that uses projected gradients. The proposed algorithm employs some extra modifications that make the method more suitable for classification tasks. The usefulness of the proposed technique to frontal face verification and facial expression recognition problems is demonstrated.

In this chapter we discuss how to learn an optimal manifold presentation to regularize nonegative matrix factorization (NMF) for data representation problems. NMF, which tries to represent a nonnegative data matrix as a product of two low rank nonnegative matrices, has been a popular method for data representation due to its ability to explore the latent part-based structure of data. Recent study shows that lots of data distributions have manifold structures, and we should respect the manifold structure when the data are represented. Recently, manifold regularized NMF used a nearest neighbor graph to regulate the learning of factorization parameter matrices and has shown its advantage over traditional NMF methods for data representation problems. However, how to construct an optimal graph to present the manifold properly remains a difficult problem due to the graph model selection, noisy features, and nonlinear distributed data. In this chapter, we introduce three effective methods to solve these problems of graph construction for manifold regularized NMF. Multiple graph learning is proposed to solve the problem of graph model selection, adaptive graph learning via feature selection is proposed to solve the problem of constructing a graph from noisy features, while multi-kernel learning-based graph construction is used to solve the problem of learning a graph from nonlinearly distributed data.

2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), 2008

Nonnegative matrix factorization (NMF) is a widely-used method for multivariate analysis of nonnegative data, the goal of which is decompose a data matrix into a basis matrix and an encoding variable matrix with all of these matrices allowed to have only nonnegative elements. In this paper we present simple algorithms for orthogonal NMF, where orthogonality constraints are imposed on basis matrix or encoding matrix. We develop multiplicative updates directly from the true gradient (natural gradient) in Stiefel manifold, whereas existing algorithms consider additive orthogonality constraints. Numerical experiments on face image data for a image representation task show that our orthogonal NMF algorithm preserves the orthogonality, while the goodness-offit (GOF) is minimized. We also apply our orthogonal NMF to a clustering task, showing that it works better than the original NMF, which is confirmed by experiments on several UCI repository data sets.

Chinese Science Bulletin, 2006

Matrix factorization is an effective tool for large-scale data processing and analysis. Nonnegative matrix factorization (NMF) method, which decomposes the nonnegative matrix into two nonnegative factor matrices, provides a new way for matrix factorization. NMF is significant in intelligent information processing and pattern recognition. This paper firstly introduces the basic idea of NMF and some new relevant methods. Then we discuss the loss functions and relevant algorithms of NMF in the framework of probabilistic models based on our researches, and the relationship between NMF and information processing of perceptual process. Finally, we make use of NMF to deal with some practical questions of pattern recognition and point out some open problems for NMF.

Applied Mathematics and Computation, 2011

Due to the extensive applications of nonnegative matrix factorizations (NMFs) of nonnegative matrices, such as in image processing, text mining, spectral data analysis, speech processing, etc., algorithms for NMF have been studied for years. In this paper, we propose a new algorithm for NMF, which is based on an alternating projected gradient (APG) approach. In particular, no zero entries appear in denominators in our algorithm which implies no breakdown occurs, and even if some zero entries appear in numerators new updates can always be improved in our algorithm. It is shown that the effect of our algorithm is better than that of Lee and Seung's algorithm when we do numerical experiments on two known facial databases and one iris database.