Bayesian principal component analysis

EUREKA: Physics and Engineering

The recent trends in collecting huge datasets have posed a great challenge that is brought by the high dimensionality and aggravated by the presence of irrelevant dimensions. Machine learning models for regression is recognized as a convenient way of improving the estimation for empirical models. Popular machine learning models is support vector regression (SVR). However, the usage of principal component analysis (PCA) as a variable reduction method along with SVR is suggested. The principal component analysis helps in building a predictive model that is simple as it contains the smallest number of variables and efficient. In this paper, we investigate the competence of SVR with PCA to explore its performance for a more accurate estimation. Simulation study and Renal Failure (RF) data of SVR optimized by four different kernel functions; linear, polynomial, radial basis, and sigmoid functions using R software, version (R x64 3.2.5) to compare the behavior of ε SVR and v-SVR models fo...

Neural Processing Letters, 2009

The Mixture of Probabilistic Principal Components Analyzers (MPPCA) is a multivariate analysis technique which defines a Gaussian probabilistic model at each unit. The numbers of units and principal directions in each unit are not learned in the original approach. Variational Bayesian approaches have been proposed for this purpose, which rely on assumptions on the probability distributions of the MPPCA parameters.

Industrial & Engineering Chemistry Research, 2014

Multivariate statistical projection methods such as principal component analysis (PCA) are the most common strategy for process monitoring in wastewater treatment plants (WWTPs). Such monitoring strategies can indeed recognize faults and achieve better control performance for fully observed data sets but can be more difficult in the case of having missing data. This study presents a variational Bayesian PCA (VBPCA) based methodology for fault detection in the WWTPs. This methodology not only is robust against missing data but also reconstructs missing data. Furthermore, a novel historical data preprocess method is proposed to deal with diurnal behaviors in the WWTPs with fast sample rate. These methodologies have been validated by process data collected from two WWTPs with different process characteristics and different sample rates. The results showed that the proposed methodology is capable of detecting sensor faults and process faults with good accuracy under different scenarios (highly and lowly instrumented WWTP).

Dimensionality reduction is one of the preprocessing steps in many machine learning applications and it is used to transform the features into a lower dimension space. Principal Component Analysis (PCA) technique is one of the most famous unsupervised dimensionality reduction techniques. The goal of the PCA is to find the space, which represents the direction of the maximum variance of the given data. This paper highlights the basic background needed to understand and implement the PCA technique. This paper starts with basic definitions of the PCA technique and the algorithms of two methods of calculating PCA, namely, the covariance matrix and Singular Value Decomposition (SVD) methods. Moreover, a number of numerical examples are illustrated to show how the PCA space is calculated in easy steps. Three experiments are conducted to show how to apply PCA in the real applications including biometrics, image compression, and visualization of high-dimensional datasets.

SSRN Electronic Journal, 2019

Nowadays, dealing with high dimension data has become common in many fields such as network security, image processing, medical diagnosis, biometric systems, e-commerce, etc. In data base with higher dimension the direct use of data is costlier in terms of its storage, processing, mathematical computations etc. Moreover, in many applications with wireless networks, Received Signal Strength (RSS) needs to be processed to remove the redundant readings. In this paper, we discussed Principal Component Analysis (PCA) as a dimensionality reduction technique. PCA can be done either by calculating the covariance matrix or by Singular Value Decomposition (SVD). We used covariance matrix calculation due to its less complexity. Simulation results verify that the correlated input data sequence is sorted to uncorrelated principal components (PC's) with decreasing variation and some of first PC's can represent nearly all data sets.

Principal component analysis (PCA) is a multivariate technique that analyzes a data table in which observations are described by several inter-correlated quantitative dependent variables. Its goal is to extract the important information from the table, to represent it as a set of new orthogonal variables called principal components, and to display the pattern of similarity of the observations and of the variables as points in maps. The quality of the PCA model can be evaluated using cross-validation techniques such as the bootstrap and the jackknife. PCA can be generalized as correspondence analysis (CA) in order to handle qualitative variables and as multiple factor analysis (MFA) in order to handle heterogeneous sets of variables. Mathematically, PCA depends upon the eigen-decomposition of positive semidefinite matrices and upon the singular value decomposition (SVD) of rectangular matrices.

Journal of Computational and Graphical Statistics, 2017

We discuss the problem of estimating the number of principal components in Principal Components Analysis (PCA). Despite of the importance of the problem and the multitude of solutions proposed in the literature, it comes as a surprise that there does not exist a coherent asymptotic framework which would justify different approaches depending on the actual size of the data set. In this paper we address this issue by presenting an approximate Bayesian approach based on Laplace approximation and introducing a general method for building the model selection criteria, called PEnalized SEmi-integrated Likelihood (PESEL). Our general framework encompasses a variety of existing approaches based on probabilistic models, like e.g. Bayesian Information Criterion for the Probabilistic PCA (PPCA), and allows for construction of new criteria, depending on the size of the data set at hand and additional prior information. Specifically, we apply PESEL to derive two new criteria for data sets where the number of variables substantially exceeds the number of observations, which is out of the scope of currently existing approaches. We also report results of extensive simulation studies and real data analysis, which illustrate good properties of our proposed criteria as compared to the state-of-the-art methods and very recent proposals. Specifically, these simulations show that PESEL based criteria can be quite robust against deviations from the probabilistic model assumptions. Selected PESEL based criteria for the estimation of the number of principal components are implemented in R package varclust, which is available on github (https://github.com/psobczyk/varclust).

Procedia Computer Science, 2019

Research in the fields of machine learning and intelligent systems addresses essential problem of developing computer algorithms that can deal with huge amounts of data and then utilize this data in an intellectual way to solve a variety of real-world problems. In many applications, to interpret data with a large number of variables in a meaningful way, it is essential to reduce the number of variables and interpret linear combinations of the data. Principal Component Analysis (PCA) is an unsupervised learning technique that uses sophisticated mathematical principles to reduce the dimensionality of large datasets. The goal of this paper is to provide a complete understanding of the sophisticated PCA in the fields of machine learning and data dimensional reduction. It explains its mathematical aspect and describes its relationship with Singular Value Decomposition (SVD) when PCA is calculated using the covariance matrix. In addition, with the use of MATLAB, the paper shows the usefulness of PCA in representing and visualizing Iris dataset using a smaller number of variables.

2021

Multivariate Statistical Process Control (MSPC) seeks to monitor several quality characteristics simultaneously. However, it has limitations derived from its inability to identify the source of special variation in the process. In this research, a proposed model that does not have this limitation is presented. In this paper, data from two scenarios were used: (A) data created by simulation and (B) random variable data obtained from the analysed product, which in this case corresponds to cheese production slicing process in the dairy industry. The model includes a dimensional reduction procedure based on the centrality and data dispersion. The goal is to recognise a multivariate pattern from the conjunction of univariate variables with variation patterns so that the model indicates the univariate patterns from the multivariate pattern. The model consists of two stages. The first stage is concerned with the identification process and uses Moving Windows (MWs) for data segmentation and...

Chemometrics and Intelligent Laboratory Systems

 Users may download and print one copy of any publication from the public portal for the purpose of private study or research.  You may not further distribute the material or use it for any profit-making activity or commercial gain  You may freely distribute the URL identifying the publication in the public portal If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.