Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Canonical Correlation Analysis
Finding the Positions and the Images in Cca
Extensions of Canonical Correlation Analysis
Discussion
References

A Tutorial on Canonical Correlation Methods

Profile image of Viivi UurtioViivi Uurtio

ACM Computing Surveys

https://doi.org/10.1145/3136624
visibility

…

description

36 pages

link

1 file

Abstract

Canonical correlation analysis is a family of multivariate statistical methods for the analysis of paired sets of variables. Since its proposition, canonical correlation analysis has, for instance, been extended to extract relations between two sets of variables when the sample size is insufficient in relation to the data dimensionality, when the relations have been considered to be non-linear, and when the dimensionality is too large for human interpretation. This tutorial explains the theory of canonical correlation analysis, including its regularised, kernel, and sparse variants. Additionally, the deep and Bayesian CCA extensions are briefly reviewed. Together with the numerical examples, this overview provides a coherent compendium on the applicability of the variants of canonical correlation analysis. By bringing together techniques for solving the optimisation problems, evaluating the statistical significance and generalisability of the canonical correlation model, and interpr...

Related papers

Semi-Paired Probabilistic Canonical Correlation Analysis
博 张

CCA is a powerful tool for analyzing paired multi-view data. However, when facing semi-paired multi-view data which widely exist in real-world problems, CCA usually performs poorly due to its requirement of data pairing between different views in nature. To cope with this problem, we propose a semi-paired variant of CCA named SemiPCCA based on the probabilistic model for CCA. Experiments with artificially generated samples demonstrate the effectiveness of the proposed method.

View PDFchevron_right
Sparsification of Probabilistic Canonical Correlation Analysis
Daniel Livingstone

2009

Abstract We have recently developed several ways of performing Canonical Correlation Analysis [1, 5, 7, 4] with probabilistic methods rather than the standard statistical tools. However, the computational demands of training such methods scales with the square of the number of samples, making these methods uncompetitive with eg artificial neural network methods [3, 2].

View PDFchevron_right
Sparse canonical correlation analysis
Michael Saunders

arXiv (Cornell University), 2017

Canonical correlation analysis was proposed by Hotelling [6] and it measures linear relationship between two multidimensional variables. In high dimensional setting, the classical canonical correlation analysis breaks down. We propose a sparse canonical correlation analysis by adding 1 constraints on the canonical vectors and show how to solve it efficiently using linearized alternating direction method of multipliers (ADMM) and using TFOCS as a black box. We illustrate this idea on simulated data.

View PDFchevron_right
An improved method for generalized constrained canonical correlation analysis
Yoshio Takane

Computational Statistics & Data Analysis, 2006

An improved method for generalized constrained canonical correlation analysis (GC-CANO) is proposed. In the original GCCANO, data matrices were first decomposed into the sum of several matrices according to some external information on rows and columns of the data matrices. Decomposed matrices were then subjected to canonical correlation analysis (CANO). However, orthogonal decompositions of data matrices do not necessarily entail orthogonal decompositions of projectors defined by the data matrices. This latter property is crucial in additive partitionings of the total association between two sets of variables. Consequently, no additive partitionings of the total association was possible in the original GCCANO. In this paper two orthogonal decompositions of projectors were proposed that allow additive partitionings of the total association. Terms in the decompositions have straightforward interpretations. An improved method for GCCANO is developed based on the decompositions, while preserving the most important features of the original method. An example is given to illustrate the proposed method. (Yoshio Takane), yanai@rd.dnc.ac.jp (Haruo Yanai), heungsun.hwang@hec.ca (Heungsun Hwang).

View PDFchevron_right
Multiset Canonical Correlation Analysis
Jan De Leeuw

The Burt matrix collects all bivariate cross tables, and/or covariance matrices, of m variables in a single matrix. Various forms of canonical analysis based on the Burt matrix are discussed.

View PDFchevron_right
A Probabilistic Interpretation of Canonical Correlation Analysis
Michael Jordan

2005

We give a probabilistic interpretation of canonical correlation (CCA) analysis as a latent variable model for two Gaussian random vectors. Our interpretation is similar to the probabilistic interpretation of principal component analysis Bishop, 1999, Roweis, 1998). In addition, we cast Fisher linear discriminant analysis (LDA) within the CCA framework.

View PDFchevron_right
A robust predictive approach for canonical correlation analysis
Jorge Gabriel Adrover

Journal of Multivariate Analysis, 2015

Canonical correlation analysis (CCA) is a dimension-reduction technique in which two random vectors from high dimensional spaces are reduced to a new pair of low dimensional vectors after applying linear transformations to each of them, retaining as much information as possible. The components of the transformed vectors are called canonical variables. One seeks linear combinations of the original vectors maximizing the correlation subject to the constraint that they are to be uncorrelated with the previous canonical variables within each vector. By these means one actually gets two transformed random vectors of lower dimension whose expected square distance has been minimized subject to have uncorrelated components of unit variance within each vector. Since the closeness between the two transformed vectors is evaluated through a highly sensitive measure to outlying observations as the mean square loss, the linear transformations we are seeking are also affected. In this paper we use a robust univariate dispersion measure (like an M-scale) based on the distance of the transformed vectors to derive robust Sestimators for canonical vectors and correlations. An iterative algorithm is performed by exploiting the existence of efficient algorithms for S-estimation in the context of Principal Component Analysis. Some convergence properties are analyzed for the iterative algorithm. A simulation study is conducted to compare the new procedure with some other robust competitors available in the literature, showing a remarkable performance. We also prove that the proposal is Fisher consistent.

View PDFchevron_right
Two-Dimensional Canonical Correlation Analysis
Seungjin Choi

IEEE Signal Processing Letters, 2007

In this letter we present a method of twodimensional canonical correlation analysis (2D-CCA) where we extend the standard CCA in such a way that relations between two different sets of image data are directly sought without reshaping images into vectors. We stess that 2D-CCA dramatically reduces the computational complexity, compared to the standard CCA. We show the useful behavior of 2D-CCA through numerical examples of correspondence learning between face images in different poses and illumination conditions.

View PDFchevron_right
Statistical Consistency of Kernel Canonical Correlation Analysis
Arthur Gretton

The Journal of Machine Learning …, 2007

View PDFchevron_right
Compositional Canonical Correlation Analysis
Juan José Egozcue

The study of the relationships between two compositions by means of canonical correlation analysis is addressed. A compositional version of canonical correlation analysis is developed, and called CODA-CCO. We consider two approaches, using the centred log-ratio transformation and the calculation of all possible pairwise log-ratios within sets. The relationships between both approaches are pointed out, and their merits are discussed. The related covariance matrices are structurally singular, and this is efficiently dealt with by using generalized inverses. We develop compositional canonical biplots and detail their properties. The canonical biplots are shown to be powerful tools for discovering the most salient relationships between two compositions. Some guidelines for compositional canonical biplots construction are discussed. A geological data set with X-ray fluorescence spectrometry measurements on major oxides and trace elements is used to illustrate the proposed method. The rel...

View PDFchevron_right

References (92)

  1. S Akaho. 2001. A Kernel Method For Canonical Correlation Analysis. In In Proceedings of the International Meeting of the Psychometric Society (IMPS2001.
  2. Md A Alam, M Nasser, and K Fukumizu. 2008. Sensitivity analysis in robust and kernel canonical correla- tion analysis. In Computer and Information Technology, 2008. ICCIT 2008. 11th International Confer- ence on. IEEE, 399-404.
  3. TW Anderson. 2003. An introduction to statistical multivariate analysis. (2003).
  4. G Andrew, R Arora, J Bilmes, and K Livescu. 2013. Deep canonical correlation analysis. In International Conference on Machine Learning. 1247-1255.
  5. C Archambeau and FR Bach. 2009. Sparse probabilistic projections. In Advances in neural information processing systems. 73-80.
  6. C Archambeau, N Delannay, and M Verleysen. 2006. Robust probabilistic projections. In Proceedings of the 23rd International conference on machine learning. ACM, 33-40.
  7. S Arlot, A Celisse, and others. 2010. A survey of cross-validation procedures for model selection. Statistics surveys 4 (2010), 40-79.
  8. F Bach, R Jenatton, J Mairal, G Obozinski, and others. 2011. Convex optimization with sparsity-inducing norms. Optimization for Machine Learning 5 (2011).
  9. FR Bach and MI Jordan. 2002. Kernel independent component analysis. Journal of machine learning re- search 3, Jul (2002), 1-48.
  10. FR Bach and MI Jordan. 2005. A probabilistic interpretation of canonical correlation analysis. (2005).
  11. MS Bartlett. 1938. Further aspects of the theory of multiple regression. In Mathematical Proceedings of the Cambridge Philosophical Society, Vol. 34. Cambridge Univ Press, 33-40.
  12. MS Bartlett. 1941. The statistical significance of canonical correlations. Biometrika 32, 1 (1941), 29-37.
  13. B Baur and S Bozdag. 2015. A canonical correlation analysis-based dynamic bayesian network prior to infer gene regulatory networks from multiple types of biological data. Journal of Computational Biology 22, 4 (2015), 289-299.
  14. Å Björck and GH Golub. 1973. Numerical methods for computing angles between linear subspaces. Mathe- matics of computation 27, 123 (1973), 579-594.
  15. MB Blaschko, CH Lampert, and A Gretton. 2008. Semi-supervised laplacian regularization of kernel canon- ical correlation analysis. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 133-145.
  16. MW Browne. 2000. Cross-validation methods. Journal of mathematical psychology 44, 1 (2000), 108-132.
  17. E Burg and J Leeuw. 1983. Non-linear canonical correlation. British journal of mathematical and statistical psychology 36, 1 (1983), 54-80.
  18. J Cai. 2013. The distance between feature subspaces of kernel canonical correlation analysis. Mathematical and Computer Modelling 57, 3 (2013), 970-975.
  19. 1:30 L Cao, Z Ju, J Li, R Jian, and C Jiang. 2015. Sequence detection analysis based on canonical correlation for steady-state visual evoked potential brain computer interfaces. Journal of neuroscience methods 253 (2015), 10-17.
  20. JD Carroll. 1968. Generalization of canonical correlation analysis to three or more sets of variables. In Proceedings of the 76th annual convention of the American Psychological Association, Vol. 3. 227-228.
  21. B Chang, U Kr üger, R Kustra, and J Zhang. 2013. Canonical Correlation Analysis based on Hilbert-Schmidt Independence Criterion and Centered Kernel Target Alignment.. In ICML (2). 316-324.
  22. X Chen, S Chen, H Xue, and X Zhou. 2012. A unified dimensionality reduction framework for semi-paired and semi-supervised multi-view data. Pattern Recognition 45, 5 (2012), 2005-2018.
  23. X Chen, C He, and H Peng. 2014. Removal of muscle artifacts from single-channel EEG based on ensemble empirical mode decomposition and multiset canonical correlation analysis. Journal of Applied Mathe- matics 2014 (2014).
  24. X Chen, H Liu, and JG Carbonell. 2012. Structured sparse canonical correlation analysis. In International Conference on Artificial Intelligence and Statistics. 199-207.
  25. A Cichonska, J Rousu, P Marttinen, AJ Kangas, P Soininen, T Lehtim äki, OT Raitakari, M-R J ärvelin, V Salomaa, M Ala-Korpela, and others. 2016. metaCCA: Summary statistics-based multivariate meta- analysis of genome-wide association studies using canonical correlation analysis. Bioinformatics (2016), btw052.
  26. N Cristianini, J Shawe-Taylor, and H Lodhi. 2002. Latent semantic kernels. Journal of Intelligent Informa- tion Systems 18, 2-3 (2002), 127-152.
  27. R Cruz-Cano and MLT Lee. 2014. Fast regularized canonical correlation analysis. Computational Statistics & Data Analysis 70 (2014), 88-100.
  28. J Dauxois and GM Nkiet. 1997. Canonical analysis of two Euclidean subspaces and its applications. Linear Algebra Appl. 264 (1997), 355-388.
  29. DL Donoho and IM Johnstone. 1995. Adapting to unknown smoothness via wavelet shrinkage. Journal of the american statistical association 90, 432 (1995), 1200-1224.
  30. RB Dunham and DJ Kravetz. 1975. Canonical correlation analysis in a predictive system. The Journal of Experimental Education 43, 4 (1975), 35-42.
  31. C Eckart and G Young. 1936. The approximation of one matrix by another of lower rank. Psychometrika 1, 3 (1936), 211-218.
  32. B Efron. 1979. Computers and the theory of statistics: thinking the unthinkable. SIAM review 21, 4 (1979), 460-480.
  33. LM Ewerbring and FT Luk. 1989. Canonical correlations and generalized SVD: applications and new algo- rithms. In 32nd Annual Technical Symposium. International Society for Optics and Photonics, 206-222.
  34. J Fang, D Lin, SC Schulz, Z Xu, VD Calhoun, and Y-P Wang. 2016. Joint sparse canonical correlation analysis for detecting differential imaging genetics modules. Bioinformatics 32, 22 (2016), 3480-3488.
  35. Y Fujikoshi and LG Veitch. 1979. Estimation of dimensionality in canonical correlation analysis. Biometrika 66, 2 (1979), 345-351.
  36. K Fukumizu, FR Bach, and A Gretton. 2007. Statistical consistency of kernel canonical correlation analysis. Journal of Machine Learning Research 8, Feb (2007), 361-383.
  37. C Fyfe and PL Lai. 2000. Canonical correlation analysis neural networks. In Pattern Recognition, 2000. Proceedings. 15th International Conference on, Vol. 2. IEEE, 977-980.
  38. GH Golub and CF Van Loan. 2012. Matrix computations. Vol. 3. JHU Press.
  39. GH Golub and H Zha. 1995. The canonical correlations of matrix pairs and their numerical computation. In Linear algebra for signal processing. Springer, 27-49.
  40. I Gonz ález, S Déjean, PGP Martin, O Gonc ¸alves, P Besse, and A Baccini. 2009. Highlighting relationships between heterogeneous biological data through graphical displays based on regularized canonical cor- relation analysis. Journal of Biological Systems 17, 02 (2009), 173-199.
  41. BK Gunderson and RJ Muirhead. 1997. On estimating the dimensionality in canonical correlation analysis. Journal of Multivariate Analysis 62, 1 (1997), 121-136.
  42. DR Hardoon, J Mourao-Miranda, M Brammer, and J Shawe-Taylor. 2007. Unsupervised analysis of fMRI data using kernel canonical correlation. NeuroImage 37, 4 (2007), 1250-1259.
  43. DR Hardoon and J Shawe-Taylor. 2009. Convergence analysis of kernel canonical correlation analysis: the- ory and practice. Machine learning 74, 1 (2009), 23-38.
  44. DR Hardoon and J Shawe-Taylor. 2011. Sparse canonical correlation analysis. Machine Learning 83, 3 (2011), 331-353.
  45. DR Hardoon, S Szedmak, and J Shawe-Taylor. 2004. Canonical correlation analysis: An overview with ap- plication to learning methods. Neural computation 16, 12 (2004), 2639-2664.
  46. MJR Healy. 1957. A rotation method for computing canonical correlations. Math. Comp. 11, 58 (1957), 83- 86.
  47. C Heij and B Roorda. 1991. A modified canonical correlation approach to approximate state space modelling. In Decision and Control, 1991., Proceedings of the 30th IEEE Conference on. IEEE, 1343-1348.
  48. AE Hoerl and RW Kennard. 1970. Ridge regression: Biased estimation for nonorthogonal problems. Techno- metrics 12, 1 (1970), 55-67.
  49. JW Hooper. 1959. Simultaneous equations and canonical correlation theory. Econometrica: Journal of the Econometric Society (1959), 245-256.
  50. CE Hopkins. 1969. Statistical analysis by canonical correlation: a computer application. Health services research 4, 4 (1969), 304.
  51. P Horst. 1961. Relations among sets of measures. Psychometrika 26, 2 (1961), 129-149.
  52. H Hotelling. 1935. The most predictable criterion. Journal of educational Psychology 26, 2 (1935), 139.
  53. H Hotelling. 1936. Relations between two sets of variates. Biometrika 28, 3/4 (1936), 321-377.
  54. WW Hsieh. 2000. Nonlinear canonical correlation analysis by neural networks. Neural Networks 13, 10 (2000), 1095-1105.
  55. I Huopaniemi, T Suvitaival, J Nikkil ä, M Orešič, and S Kaski. 2010. Multivariate multi-way analysis of multi-source data. Bioinformatics 26, 12 (2010), i391-i398.
  56. A Kabir, RD Merrill, AA Shamim, RDW Klemn, AB Labrique, P Christian, KP West Jr, and M Nasser. 2014. Canonical correlation analysis of infant's size at birth and maternal factors: a study in rural Northwest Bangladesh. PloS one 9, 4 (2014), e94243.
  57. M Kang, B Zhang, X Wu, C Liu, and J Gao. 2013. Sparse generalized canonical correlation analysis for biological model integration: a genetic study of psychiatric disorders. In Engineering in Medicine and Biology Society (EMBC), 2013 35th Annual International Conference of the IEEE. IEEE, 1490-1493.
  58. JR Kettenring. 1971. Canonical analysis of several sets of variables. Biometrika (1971), 433-451.
  59. A Kimura, M Sugiyama, T Nakano, H Kameoka, H Sakano, E Maeda, and K Ishiguro. 2013. SemiCCA: Efficient semi-supervised learning of canonical correlations. Information and Media Technologies 8, 2 (2013), 311-318.
  60. A Klami and S Kaski. 2007. Local dependent components. In Proceedings of the 24th international conference on Machine learning. ACM, 425-432.
  61. A Klami, S Virtanen, and S Kaski. 2012. Bayesian exponential family projections for coupled data sources. arXiv preprint arXiv:1203.3489 (2012).
  62. A Klami, S Virtanen, and S Kaski. 2013. Bayesian canonical correlation analysis. Journal of Machine Learn- ing Research 14, Apr (2013), 965-1003.
  63. D Krstajic, LJ Buturovic, DE Leahy, and S Thomas. 2014. Cross-validation pitfalls when selecting and assessing regression and classification models. Journal of cheminformatics 6, 1 (2014), 1.
  64. PL Lai and C Fyfe. 1999. A neural implementation of canonical correlation analysis. Neural Networks 12, 10 (1999), 1391-1397.
  65. PL Lai and C Fyfe. 2000. Kernel and nonlinear canonical correlation analysis. International Journal of Neural Systems 10, 05 (2000), 365-377.
  66. NB Larson, GD Jenkins, MC Larson, RA Vierkant, TA Sellers, CM Phelan, JM Schildkraut, R Sutphen, PPD Pharoah, S A Gayther, and others. 2014. Kernel canonical correlation analysis for assessing gene- gene interactions and application to ovarian cancer. European Journal of Human Genetics 22, 1 (2014), 126-131.
  67. SC Larson. 1931. The shrinkage of the coefficient of multiple correlation. Journal of Educational Psychology 22, 1 (1931), 45.
  68. H-S Lee. 2007. Canonical correlation analysis using small number of samples. Communications in Statistic- sSimulation and Computation R 36, 5 (2007), 973-985.
  69. SE Leurgans, RA Moyeed, and BW Silverman. 1993. Canonical correlation analysis when the data are curves. Journal of the Royal Statistical Society. Series B (Methodological) (1993), 725-740.
  70. H Lindsey, JT Webster, and S Halpern. 1985. Canonical Correlation as a Discriminant Tool in a Periodontal Problem. Biometrical journal 27, 3 (1985), 257-264.
  71. P Marttinen, J Gillberg, A Havulinna, J Corander, and S Kaski. 2013. Genome-wide association studies with high-dimensional phenotypes. Statistical applications in genetics and molecular biology 12, 4 (2013), 413-431.
  72. XM Tu. 1991. A bootstrap resampling scheme for using the canonical correlation technique in rank estima- tion. Journal of chemometrics 5, 4 (1991), 333-343.
  73. XM Tu, DS Burdick, DW Millican, and LB McGown. 1989. Canonical correlation technique for rank estima- tion of excitation-emission matrixes. Analytical Chemistry 61, 19 (1989), 2219-2224.
  74. V Uurtio, M Bomberg, K Nybo, M It ävaara, and J Rousu. 2015. Canonical correlation methods for exploring microbe-environment interactions in deep subsurface. In International Conference on Discovery Science. Springer, 299-307.
  75. JP Van de Geer. 1984. Linear relations amongk sets of variables. Psychometrika 49, 1 (1984), 79-94.
  76. T Van Gestel, JAK Suykens, J De Brabanter, B De Moor, and J Vandewalle. 2001. Kernel canonical cor- relation analysis and least squares support vector machines. In International Conference on Artificial Neural Networks. Springer, 384-389.
  77. HD Vinod. 1976. Canonical ridge and econometrics of joint production. Journal of Econometrics 4, 2 (1976), 147-166.
  78. S Waaijenborg, PC Verselewel de Witt Hamer, and AH Zwinderman. 2008. Quantifying the association between gene expressions and DNA-markers by penalized canonical correlation analysis. Statistical Applications in Genetics and Molecular Biology 7, 1 (2008).
  79. C Wang. 2007. Variational Bayesian approach to canonical correlation analysis. IEEE Transactions on Neu- ral Networks 18, 3 (2007), 905-910.
  80. D Wang, L Shi, DS Yeung, and ECC Tsang. 2005. Nonlinear canonical correlation analysis of fMRI signals using HDR models. In 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. IEEE, 5896-5899.
  81. GC Wang, N Lin, and B Zhang. 2013. Dimension reduction in functional regression using mixed data canon- ical correlation analysis. Stat Interface 6 (2013), 187-196.
  82. DS Watkins. 2004. Fundamentals of matrix computations. Vol. 64. John Wiley & Sons.
  83. FV Waugh. 1942. Regressions between sets of variables. Econometrica, Journal of the Econometric Society (1942), 290-310.
  84. DM Witten, R Tibshirani, and T Hastie. 2009. A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics (2009), kxp008.
  85. KW Wong, PCW Fung, and CC Lau. 1980. Study of the mathematical approximations made in the basis- correlation method and those made in the canonical-transformation method for an interacting Bose gas. Physical Review A 22, 3 (1980), 1272.
  86. T Yamada and T Sugiyama. 2006. On the permutation test in canonical correlation analysis. Computational statistics & data analysis 50, 8 (2006), 2111-2123.
  87. H Yamamoto, H Yamaji, E Fukusaki, H Ohno, and H Fukuda. 2008. Canonical correlation analysis for mul- tivariate regression and its application to metabolic fingerprinting. Biochemical Engineering Journal 40, 2 (2008), 199-204.
  88. Y-H Yuan, Q-S Sun, and H-W Ge. 2014. Fractional-order embedding canonical correlation analysis and its applications to multi-view dimensionality reduction and recognition. Pattern Recognition 47, 3 (2014), 1411-1424.
  89. Y-H Yuan, Q-S Sun, Q Zhou, and D-S Xia. 2011. A novel multiset integrated canonical correlation analysis framework and its application in feature fusion. Pattern Recognition 44, 5 (2011), 1031-1040.
  90. B Zhang, J Hao, G Ma, J Yue, and Z Shi. 2014. Semi-paired probabilistic canonical correlation analysis. In International Conference on Intelligent Information Processing. Springer, 1-10.
  91. H Zou and T Hastie. 2005. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67, 2 (2005), 301-320.
  92. Received February 2017; revised -; accepted - ACM Computing Surveys, Vol. 1, No. 1, Article 1, Publication date: February 2017.

Related papers

Regularized Generalized Canonical Correlation Analysis
abdi hanafi

Psychometrika, 2011

Regularized generalized canonical correlation analysis (RGCCA) is a generalization of regularized canonical correlation analysis to three or more sets of variables. It constitutes a general framework for many multi-block data analysis methods. It combines the power of multi-block data analysis methods (maximization of well identified criteria) and the flexibility of PLS path modeling (the researcher decides which blocks are connected and which are not). Searching for a fixed point of the stationary equations related to RGCCA, a new monotonically convergent algorithm, very similar to the PLS algorithm proposed by Herman Wold, is obtained. Finally, a practical example is discussed.

View PDFchevron_right
Sparse canonical correlation analysis: New formulation and algorithm
Delin Chu

IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013

In this paper, we study canonical correlation analysis (CCA), which is a powerful tool in multivariate data analysis for finding the correlation between two sets of multi-dimensional variables. The main contributions of the paper are: (i) to reveal the equivalent relationship between a recursive formula and a trace formula for the multiple CCA problem; (ii) to obtain the explicit characterization for all solutions of the multiple CCA problem even when the corresponding covariance matrices are singular; (iii) to develop a new sparse CCA algorithm; and (iv) to establish the equivalent relationship between the uncorrelated linear discriminant analysis and the CCA problem. We test several simulated and real world data sets in gene classification and crosslanguage document retrieval to demonstrate the effectiveness of the proposed algorithm. The performance of the proposed method is competitive with the state-of-the-art sparse CCA algorithms.

View PDFchevron_right
Regularized multiple-set canonical correlation analysis
Yoshio Takane

2008

Multiple-set canonical correlation analysis (Generalized CANO or GCANO for short) is an important technique because it subsumes a number of interesting multivariate data analysis techniques as special cases. More recently, it has also been recognized as an important technique for integrating information from multiple sources. In this paper, we present a simple regularization technique for GCANO and demonstrate its usefulness. Regularization is deemed important as a way of supplementing insufficient data by prior knowledge, and/or of incorporating certain desirable properties in the estimates of parameters in the model. Implications of regularized GCANO for multiple correspondence analysis are also discussed. Examples are given to illustrate the use of the proposed technique.

View PDFchevron_right
Sparse Kernel Canonical Correlation Analysis
Delin Chu

2013

Canonical correlation analysis (CCA) is a multivariate statistical technique for finding the linear relationship between two sets of variables. The kernel generalization of CCA named kernel CCA has been proposed to find nonlinear relations between data sets. Despite the wide usage of CCA and kernel CCA, they have one common limitation that is the lack of sparsity in their solution. In this paper, we consider sparse kernel CCA and propose a novel sparse kernel CCA algorithm (SKCCA). Our algorithm is based on a relationship between kernel CCA and least squares. Sparsity of the dual transformations is introduced by penalizing the 1-norm of dual vectors. Experiments demonstrate that our algorithm not only performs well in computing sparse dual transformations but also can alleviate the over-fitting problem of kernel CCA. Index Terms-canonical correlation analysis, kernel, sparsity I. INTRODUCTION T HE description of relationship between two sets of variables has long been an interesting topic to many researchers. Canonical correlation analysis (CCA) [10] is a multivariate statistical technique for finding the linear relationship between two sets of variables. It seeks a linear transformation for each of the two sets of variables in a way that the projected variables in the transformed space are maximally correlated. In recent years, CCA has been successfully applied in various areas, including genomic data analysis [19], [20] and bilingual analysis [18], where researchers can measure multiple sets of variables on a single subject. For instance, DNA copy number variations, gene expression, and single nucleotide polymorphism (SNP) data might all be available on a common set of patient samples. Since CCA only consider linear transformation of the original variables, it fails to capture nonlinear relations. However, in a wide range of practical problems linear relations may not be adequate for studying relation among variables. Detecting nonlinear relations among data is important and useful in modern data analysis, especially when dealing with data that are not in the form of vectors, such as text documents, images, microarray data and so on. A natural extension, therefore, is to explore and exploit nonlinear relations among data. Among nonlinear extensions of CCA, one most frequently used approach is the kernel generalization

View PDFchevron_right
Representation-Constrained Canonical Correlation Analysis: A Hybridization of Canonical Correlation and Principal Component Analyses
sudhanshu mishra

2009

View PDFchevron_right

Cited by

In Situ Al-TiC Composites Fabricated by Self-propagating High-Temperature Reaction: Insights on Reaction Pathways and Their Microstructural Signatures
Alan Taub

Metallurgical and Materials Transactions A, 2020

In this study, we have analyzed the formation mechanisms and processing-microstructure relationships for Al/TiC metal matrix nanocomposites produced in situ via thermite-assisted (e.g., CuO) self-propagating high-temperature synthesis (SHS). Al/TiC composites were created by reacting Al-TiC -CuO pellets in an Al melt using a wide variety of processing conditions (e.g., precursor powder amounts, bulk melt temperature, precursor powder size, pellet packing method). As-cast composites were visualized using both 2D (SEM) and 3D (TXM) microscopy techniques, to study TiC particle and secondary precipitate (e.g., Al 3 Ti) characteristics at the nanoscale. SHS-produced samples reveal complex microstructures consisting of individual and clustered TiC particles, elongated Al 3 Ti intermetallics, and C-rich regions surrounded by TiC. Based on a thermodynamic analysis and our microstructural observations, we propose three dominant TiC formation pathways, each resulting in a distinct microstructural signature. Finally, we utilize multivariate statistics (canonical correlations) on the full breadth of imaging data to infer the dominant processing variables (i.e., amount of CuO and C) that most strongly influence TiC particle characteristics and the final composite microstructure. We also discuss how the dominant processing variables relate to the proposed formation pathways and how they may inform the rational design of future composites produced via thermite-assisted SHS.

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved