Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Conclusions
References
All Topics
Computer Science

Sparse methods for biomedical data

Profile image of Jieping YeJieping Ye

2012, ACM SIGKDD Explorations Newsletter

https://doi.org/10.1145/2408736.2408739
visibility

…

description

24 pages

link

1 file

Abstract

Following recent technological revolutions, the investigation of massive biomedical data with growing scale, diversity, and complexity has taken a center stage in modern data analysis. Although complex, the underlying representations of many biomedical data are often sparse. For example, for a certain disease such as leukemia, even though humans have tens of thousands of genes, only a few genes are relevant to the disease; a gene network is sparse since a regulatory pathway involves only a small number of genes; many biomedical signals are sparse or compressible in the sense that they have concise representations when expressed in a proper basis. Therefore, finding sparse representations is fundamentally important for scientific discovery. Sparse methods based on the '1 norm have attracted a great amount of research efforts in the past decade due to its sparsity-inducing property, convenient convexity, and strong theoretical guarantees. They have achieved great success in variou...

Related papers

Fast sparse representation approaches for the classification of high-dimensional biological data
Alioune Ngom

2012 IEEE International Conference on Bioinformatics and Biomedicine, 2012

Classifying genomic and proteomic data is very important to predict diseases in a very early stage and investigate signaling pathways. However, this poses many computationally challenging problems, such as curse of dimensionality, noise, redundancy and so on. The principle of sparse representation has been applied to analyzing high-dimensional biological data within the frameworks of clustering, classification, and dimension reduction approaches. However, the existing sparse representation approaches are either inefficient or have the difficulty of kernelization. In this paper, we propose fast activeset-based sparse coding approach and a dictionary learning framework for classifying high-dimensional biological data. We show that they can be easily kernelized. Experimental results show that our approaches are very efficient, and satisfactory accuracy can be obtained compared with existing approaches.

View PDFchevron_right
Sparse representation of data
Frank-michael Schleif

2010

The amount of electronic data available today as well as its dimensionality and complexity increases rapidly in many scientific areas including biology, (bio-)chemistry, medicine, physics and its application fields like robotics, bioinformatics or multimedia technologies. Many of these data sets are very complex but have also a simple inherent structure which allows an appropriate sparse representation and modeling of such data with less or no information loss. Advanced methods are needed to extract these inherent but hidden information. Sparsity can be observed at different levels: sparse representation of data points using e.g. dimension- ality reduction for efficient data storage, sparse representation of full data sets using e.g. prototypes to achieve compact models for lifelong learning and sparse models of the underlying data structure using sparse encoding techniques. One main goal is to achieve a human-interpretable represen- tation of the essential information. Sparse repre...

View PDFchevron_right
High-Dimensional Sparse Factor Modelling: Applications in Gene Expression Genomics
Carlos Carvalho

In studies of molecular profiling and biological pathway analysis using DNA microarray gene expression data we are utilising a broad class of sparse latent factor and regression models for large-scale multivariate analysis and regression prediction. We present examples of these applications with discussion of key aspects of the modelling and computational methodology. Our case studies are drawn from breast cancer genomics, where we are concerned with the investigation and characterisation of heterogeneity of structure related to specific oncogenic pathways, as well as predictive/prognostic uses of aggregate patterns in gene expression profiles in clinical contexts.

View PDFchevron_right
A tutorial on the Lasso approach to sparse modeling
Rasmus Bro

Chemometrics and Intelligent Laboratory Systems, 2012

In applied research data are often collected from sources with a high dimensional multivariate output. Analysis of such data is composed of e.g. extraction and characterization of underlying patterns, and often with the aim of finding a small subset of significant variables or features. Variable and feature selection is well-established in the area of regression, whereas for other types of models this seems more difficult. Penalization of the L 1 norm provides an interesting avenue for such a problem, as it produces a sparse solution and hence embeds variable selection. In this paper a brief introduction to the mathematical properties of using the L 1 norm as a penalty is given. Examples of models extended with L 1 norm penalties/constraints are presented. The examples include PCA modeling with sparse loadings which enhance interpretability of single components. Sparse inverse covariance matrix estimation is used to unravel which variables are affecting each other, and a modified PCA to model data with (piecewise) constant responses in e.g. process monitoring is shown. All examples are demonstrated on real or synthetic data. The results indicate that sparse solutions, when appropriate, can enhance model interpretability.

View PDFchevron_right
Optimization algorithms for sparse representations and applications
Andrzej Cichocki

2006

We consider the following sparse representation problem, which is called Sparse Component Analysis: identify the matrices S ∈ IR n×N and A ∈ IR m×n (m ≤ n < N ) uniquely (up to permutation of scaling), knowing only their multiplication X = AS, under some conditions, expressed either in terms of A and sparsity of S (identifiability conditions), or in terms of X (Sparse Component Analysis conditions). A crucial assumption (sparsity condition) is that S is sparse of level k in sense that each column of S has at least k nonzero elements (k = 1, 2, ..., m − 1).

View PDFchevron_right
Heuristic Approach to Sparse Approximation of Gene Regulatory Networks
Stuart Kauffman

Journal of Computational Biology, 2008

Determining the structure of the gene regulatory network using the information in genomewide profiles of mRNA abundance, such as microarray data, poses several challenges. Typically, "static" rather than dynamical profile measurements, such as those taken from steady state tissues in various conditions, are the starting point. This makes the inference of causal relationships between genes difficult. Moreover, the paucity of samples relative to the gene number leads to problems such as overfitting and underconstrained regression analysis. Here we present a novel method for the sparse approximation of gene regulatory networks that addresses these issues. It is formulated as a sparse combinatorial optimization problem which has a globally optimal solution in terms of l 0 norm error. In order to seek an approximate solution of the l 0 optimization problem, we consider a heuristic approach based on iterative greedy algorithms. We apply our method to a set of gene expression profiles comprising of 24,102 genes measured over 79 human tissues. The inferred network is a signed directed graph, hence predicts causal relationships. It exhibits typical characteristics of regulatory networks organism with partially known network topology, such as the average number of inputs per gene as well as the in-degree and out-degree distribution.

View PDFchevron_right
Sparse models for correlative and integrative analysis of imaging and genetic data
Hongbao Cao

Journal of Neuroscience Methods, 2014

The development of advanced medical imaging technologies and high-throughput genomic measurements has enhanced our ability to understand their interplay as well as their relationship with human behavior by integrating these two types of datasets. However, the high dimensionality and heterogeneity of these datasets presents a challenge to conventional statistical methods; there is a high demand for the development of both correlative and integrative analysis approaches. Here, we review our recent work on developing sparse representation based approaches to address this challenge. We show how sparse models are applied to the correlation and integration of imaging and genetic data for biomarker identification. We present examples on how these approaches are used for the detection of risk genes and classification of complex diseases such as schizophrenia. Finally, we discuss future directions on the integration of multiple imaging and genomic datasets including their interactions such as epistasis.

View PDFchevron_right
Sparse Regression
Jinseog Kim

2006

Yuan an Lin (2004) proposed the grouped LASSO, which achieves shrinkage and selection simultaneously, as LASSO does, but works on blocks of covariates. That is, the grouped LASSO provides a model where some blocks of regression coefficients are exactly zero. The grouped LASSO is useful when there are meaningful blocks of covariates such as polynomial regression and dummy variables from categorical variables. In this paper, we propose an extension of the grouped LASSO, called ‘Blockwise Sparse Regression’ (BSR). The BSR achieves shrinkage and selection simultaneously on blocks of covariates similarly to the grouped LASSO, but it works for general loss functions including generalized linear models. An efficient computational algorithm is developed and a blockwise standardization method is proposed. Simulation results show that the BSR compromises the ridge and LASSO for logistic regression. The proposed method is illustrated with two datasets.

View PDFchevron_right
Sparseness theorems and sparse representation of signals
Pando Georgiev

2004

We present general sparseness theorems showing that the solutions of various types least square and absolute value optimization problems

View PDFchevron_right
Augmented sparse reconstruction of protein signaling networks
Lance Liotta

Journal of Theoretical Biology, 2008

The problem of reconstructing and identifying intracellular protein signaling and biochemical networks is of critical importance in biology today. We sought to develop a mathematical approach to this problem using, as a test case, one of the most well-studied and clinically important signaling networks in biology today, the epidermal growth factor receptor (EGFR) driven signaling cascade. More specifically, we suggest a method, augmented sparse reconstruction, for the identification of links among nodes of ordinary differential equation (ODE) networks from a small set of trajectories with different initial conditions. Our method builds a system of representation by using a collection of integrals of all given trajectories and by attenuating block of terms in the representation itself. The system of representation is then augmented with random vectors, and l1 minimization is used to find sparse representations for the dynamical interactions of each node. Augmentation by random vectors is crucial, since sparsity alone is not able to handle the large error-in-variables in the representation. Augmented sparse reconstruction allows to consider potentially very large spaces of models and it is able to detect with high accuracy the few relevant links among nodes, even when moderate noise is added to the measured trajectories. After showing the performance of our method on a model of the EGFR protein network, we sketch briefly the potential future therapeutic applications of this approach.

View PDFchevron_right

References (127)

  1. Afonso M, Bioucas-Dias J, Figueiredo M. An augmented lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Transactions on Image Processing. 2011; 20:681-695. [PubMed: 20840899]
  2. Argyriou A, Evgeniou T, Pontil M. Convex multi-task feature learning. Machine Learning. 2008; 73(3):243-272.
  3. Bach FR. Consistency of the group lasso and multiple kernel learning. Journal of Machine Learning Research. 2008; 9:1179-1225.
  4. Bajwa W, Haupt J, Sayeed A, Nowak R. Compressive wireless sensing. International Conference on Information Processing in Sensor Networks. 2006
  5. Banerjee O, El Ghaoui L, d'Aspremont A. Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. The Journal of Machine Learning Research. 2008; 9:485-516.
  6. Baraniuk R. Compressive sensing. IEEE Signal Processing Magazine. 2007; 24(4):118-121.
  7. Belkin M, Niyogi P. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation. 2003; 15:1373-1396.
  8. Blaimer M, Breuer F, Muller M, Heidemann R, Griswold MA, Jakob PM. SMASH, SENSE, PILS, GRAPPA. Top Magn Reson Imaging. 2004; 15:223-236. [PubMed: 15548953]
  9. Bondell H, Reich B. Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with oscar. Biometrics. 2008; 64(1):115-123. [PubMed: 17608783]
  10. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning. 2011; 3:1-122.
  11. Bühlmann P. Statistical significance in high-dimensional linear models. Arxiv preprint arXiv: 1202.1377v1. 2012
  12. Busse R, Wang K, Holmes J, Brittain J, Korosec F. Optimization of variable-density cartesian sampling for time-resolved imaging. International Society for Magnetic Resonance in Medicine. 2009
  13. Candès E, Romberg. J. Quantitative robust uncertainty principles and optimally sparse decompositions. Foundations of Computational Mathematics. 2006; 6(2):227-254.
  14. Candès E, Romberg J, Tao T. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory. 2006; 52(2): 489-509.
  15. Candès E, Tao T. Near optimal signal recovery from random projections: Universal encoding strategies? IEEE Transactions on Information Theory. 2006; 52(12):5406-5425.
  16. Candès E, Wakin M. An introduction to compressive sampling. IEEE Signal Processing Magazine. 2008; 25(2):21-30.
  17. Carroll, S.; Grenier, J.; Weatherbee, S. From DNA to Diversity: Molecular Genetics and the Evolution of Animal Design. 2nd edition.. Blackwell Pub; Malden, MA: 2005.
  18. Chu W, Ghahramani Z, Falciani F, Wild D. Biomarker discovery in microarray gene expression data with gaussian processes. Bioinformatics. 2005; 21(16):3385-3393. [PubMed: 15937031]
  19. Chuang H, Lee E, Liu Y, Lee D, Ideker T. Network-based classification of breast cancer metastasis. Molecular systems biology. 2007; 3:140. [PubMed: 17940530]
  20. Chung F. Spectral Graph Theory. American Mathematical Society. 1997
  21. Cox D. Regression models and life-tables. Journal of the Royal Statistical Society. Series B (Methodological). 1972; 34(2):187-220.
  22. Danaher P, Wang P, Daniela D. The joint graphical lasso for inverse covariance estimation across multiple classes. Arxiv preprint arXiv:1111.0324. :2011.
  23. d'Aspremont A, El Ghaoui L, Jordan M, Lanckriet GRG. A direct formulation for sparse PCA using semidefinite programming. NIPS. 2005
  24. Donoho D. High-dimensional data analysis: The curses and blessings of dimensionality. 2000
  25. Donoho D. Compressed sensing. IEEE Transactions on Information Theory. 2006; 52:1289-1306.
  26. Duarte M, Davenport M, Wakin M, Baraniuk R. Sparse signal detection from incoherent projections. ICASSP. 2006
  27. Fan J, Lv J. A selective overview of variable selection in high dimensional feature space. Statistica Sinica. 2010; 20:101-148. [PubMed: 21572976]
  28. Friedman J, Hastie T, Höfling H, Tibshirani R. Pathwise coordinate optimization. Annals of Applied Statistics. 2007; 1(2):302-332.
  29. Friedman J, Hastie T, Tibshirani R. Sparse inverse covariance estimation with the graphical lasso. Biostatistics. 2008; 9(3):432-441. [PubMed: 18079126]
  30. Friedman, J.; Hastie, T.; Tibshirani, R. A note on the group lasso and a sparse group lasso. Technical report. Department of Statistics; Stanford University: 2010.
  31. Goldstein T, Osher S. The split bregman method for l1-regularized problems. SIAM Journal on Imaging Sciences. 2009; 2:323-343.
  32. Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, Bloomfield CD. Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring. Science. 1999; 286(5439):531-537. [PubMed: 10521349]
  33. Greengard L, Lee J. Accelerating the nonuniform fast fourier transform. SIAM Review. 2004; 46:443-454.
  34. Griswold MA, Jakob PM, Heidemann RM, Nittka M, Jellus V, Wang J, K. B. Haase A. Generalized autocalibrating partially parallel acquisitions (GRAPPA). Magnetic Resonance in Medicine. 2002; 47:1202-1210. [PubMed: 12111967]
  35. Griswold MA, Kannengiesser S, Heidemann RM, Wang J, Jakob PM. Field-of-view limitations in parallel imaging. Magnetic Resonance in Medicine. 2004; 52:1118-1126. [PubMed: 15508164]
  36. Guo J, Levina E, Michailidis G, Zhu J. Joint estimation of multiple graphical models. Biometrika. 2011; 98(1):1-15. [PubMed: 23049124]
  37. Guttman MA, Kellman P, Dick AJ, Lederman RJ, McVeigh ER. Real-time accelerated interactive MRI with adaptive TSENSE and UNFOLD. Magnetic Resonance in Medicine. 2003; 50:315-321. [PubMed: 12876708]
  38. Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Machine Learning. 2002; 46(1-3):389-422.
  39. Haacke, EM.; Brown, RW.; Thompson, MR.; Venkatesan, R., editors. Magnetic Resonance Imaging: Physical Principles and Sequence Design. Wiley-Liss; 1999.
  40. Hansen MS, Baltes C, Tsao J, Kozerke S, Pruessmann KP, Eggers H. k-t BLAST reconstruction from non-cartesian k-t space sampling. Magnetic Resonance in Medicine. 2006; 55:85-91. [PubMed: 16323167]
  41. Hromádka T, DeWeese M, Zador A. Sparse representation of sounds in the unanesthetized auditory cortex. PLoS Biol. 2008; 6(1):e16. [PubMed: 18232737]
  42. Huang J, Zhang T, Metaxas D. Learning with structured sparsity. Journal of Machine Learning Research. 2011; 12:3371-3412.
  43. Huang S, Li J, Sun L, Liu J, Wu T, Chen K, Fleisher A, Reiman E, Ye J. Learning brain connectivity of alzheimer's disease from neuroimaging data. NIPS, pages. 2009:808-816.
  44. Jacob L, Obozinski G, Vert J. Group lasso with overlap and graph lasso. ICML. 2009
  45. Jenatton R, Audibert J-Y, Bach F. Structured variable selection with sparsity-inducing norms. Journal of Machine Learning Research. 2011; 12:2777-2824.
  46. Jenatton R, Mairal J, Obozinski G, Bach F. Proximal methods for sparse hierarchical dictionary learning. ICML. 2010
  47. Kellman P. Parallel imaging: the basics. ISMRM Educational Course: MR Physics for Physicists. 2004
  48. Kellman P, Epstein FH, McVeigh ER. Adaptive sensitivity encoding incorporating temporal filtering (tsense). Magnetic Resonance in Medicine. 2001; 45:846-852. [PubMed: 11323811]
  49. Khare K, Hardy CJ, King KF, Turski PA, Marinelli L. Accelerated MR imaging using compressive sensing with no free parameters. Magnetic Resonance in Medicine. 2012
  50. Kim S, Xing E. Statistical estimation of correlated genome associations to a quantitative trait network. PLoS genetics. 2009; 5(8):e1000587. [PubMed: 19680538]
  51. Kim S, Xing EP. Tree-guided group lasso for multi-task regression with structured sparsity. ICML. 2010
  52. Koh K, Kim S, Boyd S. An interior-point method for large-scale l1-regularized logistic regression. Journal of Machine Learning Research. 2007; 8:1519-1555.
  53. Kolar M, Song L, Ahmed A, Xing E. Estimating time-varying networks. The Annals of Applied Statistics. 2010; 4(1):94-123.
  54. Kolar M, Xing E. On time varying undirected graphs. AISTAT. 2011
  55. Kowalski M. Sparse regression using mixed norms. Applied and Computational Harmonic Analysis. 2009; 27(3):303-324.
  56. Kumar S, Jayaraman K, Panchanathan S, Gurunathan R, Marti-Subirana A, Newfeld S. BEST: A novel computational approach for comparing gene expression patterns from early stages of Drosophila melanogaster development. Genetics. 2002; 162(4):2037-2047. [PubMed: 12524369]
  57. Lai P, Lustig M, B. A. C. V. S. S. B. P. J. A. M. Efficient L1SPIRiT reconstruction (ESPIRiT) for highly accelerated 3d volumetric MRI with parallel imaging and compressed sensing. ISMRM. 2010
  58. Lai P, Lustig M, V. S. S. B. A. C. ESPIRiT (efficient eigenvector-based l1spirit) for compressed sensing parallel imaging -theoretical interpretation and improved robustness for overlapped FOV prescription. ISMRM. 2011
  59. Larkman DJ, Nunes RG. Parallel magnetic resonance imaging. Physics in Medicine and Biology. 2007; 52:R15-55. [PubMed: 17374908]
  60. Li C, Li H. Network-constrained regularization and variable selection for analysis of genomic data. Bioinformatics. 2008; 24(9):1175-1182. [PubMed: 18310618]
  61. Liang D, Liu B, Wang J, Ying L. Accelerating SENSE using compressed sensing. Magnetic Resonance in Medicine. 2009; 62:1574-1584. [PubMed: 19785017]
  62. Liu H, Palatucci M, Zhang J. Blockwise coordinate descent procedures for the multi-task lasso, with applications to neural semantic basis discovery. ICML. 2009
  63. Liu H, Roeder K, Wasserman L. Stability approach to regularization selection (StARS) for high dimensional graphical models. NIPS. 2011
  64. Liu J, Ji S, Ye J. Multi-task feature learning via efficient ℓ 2, 1 -norm minimization. UAI. 2009
  65. Liu J, Ji S, Ye J. SLEP: Sparse Learning with Efficient Projections. Arizona State University. 2009
  66. Liu J, Rapin J, Chang T, Lefebvre A, Zenge M, Mueller E, Nadar MS. Dynamic cardiac MRI reconstruction with weighted redundant haar wavelets. ISMRM. 2012
  67. Liu J, Rapin J, Chang T, Schmitt P, Bi X, Lefebvre A, Zenge M, Mueller E, Nadar MS. Regularized reconstruction using redundant haar wavelets: A means to achieve high under- sampling factors in non-contrast-enhanced 4D MRA. ISMRM. 2012
  68. Liu J, Ye J. Efficient ℓ 1 / ℓ q norm regularization. Arxiv preprint arXiv:1009.4766v1. 2010
  69. Liu J, Ye J. Moreau-Yosida regularization for grouped tree structure learning. NIPS. 2010
  70. Loh P, Wainwright M. High-dimension regression with noisy and missing data: Provable guarantees with non-convexity. NIPS. 2011
  71. Lustig M, Donoho DL, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magnetic Resonance in Medicine. 2007; 58:1182-1195. [PubMed: 17969013]
  72. Lustig M, Lai P, Murphy M, Vasanawala S, Elad M, Zhang J, Pauly J. An eigen-vector approach to autocalibrating parallel MRI, where SENSE meets GRAPPA. ISMRM. 2011
  73. Lustig M, Pauly JM. SPIRiT: Iterative self-consistent parallel imaging reconstruction from arbitrary k-space. Magnetic Resonance in Medicine. 2010; 64:457-471. [PubMed: 20665790]
  74. Marton M, et al. Drug target validation and identification of secondary drug target effects using DNA microarrays. Nature Medicine. 1998; 4(11):1293-1301.
  75. Mazumder R, Hastie T. Exact covariance thresholding into connected components for large-scale graphical lasso. Arxiv preprint arXiv:1108.3829. :2011.
  76. Mazumder R, Hastie T. The graphical lasso: New insights and alternatives. Arxiv preprint arXiv: 1111.5479. :2011.
  77. Megason S, Fraser S. Imaging in systems biology. Cell. 2007; 130(5):784-795. [PubMed: 17803903]
  78. Meier L, Geer S, Bühlmann P. The group lasso for logistic regression. Journal of the Royal Statistical Society: Series B. 2008; 70:53-71.
  79. Meinshausen N, Bühlmann P. Stability selection. Journal of the Royal Statistical Society: Series B. 2010; 72:417-473.
  80. Negahban S, Wainwright M. Joint support recovery under high-dimensional scaling: Benefits and perils of ℓ 1 , ∞-regularization. NIPS. 2008:1161-1168.
  81. Nemirovski, A. Efficient methods in convex programming. Lecture Notes; 1994.
  82. Nesterov, Y. Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers; 2004.
  83. Nesterov, Y. Gradient methods for minimizing composite objective function. CORE; 2007.
  84. Ng A. Feature selection, ℓ 1 vs. ℓ 2 regularization, and rotational invariance. ICML. 2004
  85. Obozinski, G.; Taskar, B.; Jordan, MI. Joint covariate selection for grouped classification. Technical report. Statistics Department; UC Berkeley: 2007.
  86. Peng H. Bioimage informatics: a new area of engineering biology. Bioinformatics. 2008; 24(17): 1827-1836. [PubMed: 18603566]
  87. Peng J, Zhu J, B. A. Han W, Noh D-Y, Pol-lack JR, Wang P. Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer. Annals of Applied Statistics. 2010; 4(1):53-77.
  88. Pruessmann K, Weiger M, Scheidegger M, Boesiger P. SENSE: sensitivity endcoding for fast MRI. Magnetic Resonance in Medicine. 1999; 42:952-962. [PubMed: 10542355]
  89. Quattoni A, Carreras X, Collins M, Darrell T. An efficient projection for ℓ 1 , ∞ regularization. ICML. 2009
  90. Ramani S, Fessler JA. Parallel MR image reconstruction using augmented lagrangian methods. IEEE Transactions on Medical Imaging. 2011; 30:694-706. [PubMed: 21095861]
  91. Ryali S, Supekar K, Abrams D, Menon V. Sparse logistic regression for whole-brain classification of fMRI data. Neuroimage. 2010; 51(2):752-764. [PubMed: 20188193]
  92. Shen X, Pan W, Zhu Y. Likelihood-based selection and sharp parameter estimation. Journal of American Statistical Association. 2012; 107:223-232.
  93. Shi, J.; Yin, W.; Osher, S.; Sajda, P. A fast algorithm for large scale ℓ 1 -regularized logistic regression. Technical report, CAAM TR08-07. 2008.
  94. Shi W, Lee K, Wahba G. Detecting disease-causing genes by lasso-patternsearch algorithm. BMC Proceedings. 2007; 1(Suppl 1):S60. [PubMed: 18466561]
  95. Sodickson D, Manning W. Simultaneous acquisition of spatial harmonics (SMASH): fast imaging with radiofrequency coil arrays. Magnetic Resonance in Medicine. 1997; 38:591-603. [PubMed: 9324327]
  96. Stäadler N, Bühlmann P. Missing values: sparse inverse covariance estimation and an extension to sparse regression. Statistics and Computing. 2012; 22:219-235.
  97. Subramanian A, et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proceedings of the National Academy of Sciences of the United States of America. 2005; 102(43):15545-15550. [PubMed: 16199517]
  98. Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B. 1996; 58(1):267-288.
  99. Tibshirani R, Saunders M, Rosset S, Zhu J, Knight K. Sparsity and smoothness via the fused lasso. Journal Of The Royal Statistical Society Series B. 2005; 67(1):91-108.
  100. Tibshirani R, Wang P. Spatial smoothing and hot spot detection for cgh data using the fused lasso. Biostatistics. 2008; 9(1):18-29. [PubMed: 17513312]
  101. Tomancak P, Beaton A, Weiszmann R, Kwan E, Shu S, Lewis SE, Richards S, Ashburner M, Hartenstein V, Celniker SE, Rubin GM. Systematic determination of patterns of gene expression during Drosophila embryogenesis. Genome Biology. 2002; 3(12):1-14. research0088.
  102. Tropp A, Gilbert A, Strauss M. Algorithms for simultaneous sparse approximation: part I: Greedy pursuit. Signal Processing. 2006; 86(3):572-588.
  103. Uecker M, Hohage T, Block KT, Frahm J. Image reconstruction by regularized nonlinear inversion-joint estimation of coil sensitivities and image content. Magnetic Resonance in Medicine. 2008; 60:674-682. [PubMed: 18683237]
  104. Usman M, Prieto C, Schaeffter T, Batchelor PG. k-t group sparse: A method for accelerating dynamic MRI. Magnetic Resonance in Medicine. 2011; 66:1163-1176. [PubMed: 21394781]
  105. Vlaardingerbroek, MT.; Boer, JA., editors. Magnetic Resonance Imaging. Spinger; 2004.
  106. Witten D, Friedman J, Simon N. New insights and faster computations for the graphical lasso. Journal of Computational and Graphical Statistics. 2011; 20(4):892-900.
  107. Wu T, Chen Y, Hastie T, Sobel E, Lange K. Genome-wide association analysis by lasso penalized logistic regression. Bioinformatics. 2009; 25(6):714-721. [PubMed: 19176549]
  108. Xiang S, Shen X, Ye J. Efficient sparse group feature selection via nonconvex optimization. Arxiv preprint arXiv:1205.5075. 2012
  109. Yang, S.; Pan, Z.; Shen, X.; Wonka, P.; Ye, J. Fused multiple graphical lasso. Technical Report. Arizona State University; 2012.
  110. Yang S, Yuan L, Lai Y-C, Shen X, Wonka P, Ye J. Feature grouping and selection over an undirected graph. KDD. 2012
  111. Ye J, Farnum M, Yang E, Verbeeck R, Lobanov V, Raghavan N, Novak G, DiBernardo A, Narayan V. Sparse learning and stability selection for predicting MCI to AD conversion using baseline ADNI data. BMC Neurology. 2012; 12:46. [PubMed: 22731740]
  112. Ye X, Chen Y, Huang F. Computational acceleration for MR image reconstruction in partially parallel imaging. IEEE Transactions on Medical Imaging. 2011; 30:1055-1063. [PubMed: 20833599]
  113. Ying L, Sheng J. Joint image reconstruction and sensitivity estimation in sense (JSENSE). Magnetic Resonance in Medicine. 2007; 57:1196-1202. [PubMed: 17534910]
  114. Yuan L, Wang Y, Thompson P, Narayand V, Ye J. Multi-source feature learning for joint analysis of incomplete multiple heterogeneous neuroimaging data. NeuroImage. 2012; 61(3):622-632. [PubMed: 22498655]
  115. Yuan L, Woodard A, Ji S, Jiang Y, Zhou Z-H, Kumar S, Ye J. Learning sparse representations for fruit-fly gene expression pattern image annotation and retrieval. BMC Bioinformatics. 2012; 13:107. [PubMed: 22621237]
  116. Yuan M, Joseph VR, Zou H. Structured variable selection and estimation. Annals of Applied Statistics. 2009; 3:1738-1757.
  117. Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. Journal Of The Royal Statistical Society Series B. 2006; 68(1):49-67.
  118. Zhang C-H, Zhang S. Confidence intervals for low-dimensional parameters with high- dimensional data. Arxiv preprint arXiv:1110.2563v1. 2011
  119. Zhang T. Analysis of multi-stage convex relaxation for sparse regularization. Journal of Machine Learning Research. 2010; 11:1081-1107.
  120. Zhao P, Rocha G, Yu B. The composite absolute penalties family for grouped and hierarchical variable selection. Annals of Statistics. 2009; 37(6A):3468-3497.
  121. Zhong L, Kwok J. Efficient sparse modeling with automatic feature grouping. ICML. 2011
  122. Zhou S, Lafferty J, Wasserman L. Time varying undirected graphs. COLT. 2008
  123. Zhu J, Hastie T. Classification of gene microar-rays by penalized logistic regression. Biostatistics. 2004; 5(3):427-443. [PubMed: 15208204]
  124. Zhu J, Rosset S, Hastie T, Tibshirani R. 1-norm support vector machines. Neural Information Processing Systems. 2003:49-56.
  125. Zhu Y, Shen X, Pan W. Simultaneous grouping pursuit and feature selection in regression over an undirected graph. Preprint. 2012
  126. Zou H, Hastie T. Regularization and variable selection via the elastic net. Journal of The Royal Statistical Society Series B. 2005; 67(2):301-320.
  127. Zou H, Hastie T, Tibshirani R. Sparse principle component analysis. Journal of Computational and Graphical Statistics. 2006; 15(2):262-286.

Related papers

Sparse representation via &#x2113;<inf>1</inf>-minimization for underdetermined systems in classification of tumors with gene expression data
Miguel Argaez

2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2011

The development of cancer diagnosis models and cancer discovery from DNA microarray data are of great interest in bioinformatics and medicine. In pattern recognition and machine learning, a classification problem refers to finding an algorithm for assigning a given input data into one of several categories. Many natural signals are sparse or compressible in the sense that they have short representations when expressed in a suitable basis. Motivated by the recent successful algorithm developments for sparse signal recovery, we apply the selective nature of sparse representation to perform the above mentioned classification. In order to find such sparse representation we implement an ℓ 1-minimization algorithm. This methodology overcomes the lack of robustness with respect to outliers. In contrast to other classification algorithms, no model selection dependency is involved. The minimization algorithm is a convex relaxation-like that has been proven to efficiently recover sparse signals. To study its performance, the proposed method is applied to six tumor gene expression datasets and numerically compared with various support vector machine methods (SVM). The numerical results show that the ℓ 1-minimization algorithm proposed performs at least comparably and often better than SVMs.

View PDFchevron_right
Sparse representation via ℓ (1)-minimization for underdetermined systems in classification of tumors with gene expression data
Miguel Argaez

2011

The development of cancer diagnosis models and cancer discovery from DNA microarray data are of great interest in bioinformatics and medicine. In pattern recognition and machine learning, a classification problem refers to finding an algorithm for assigning a given input data into one of several categories. Many natural signals are sparse or compressible in the sense that they have short representations when expressed in a suitable basis. Motivated by the recent successful algorithm developments for sparse signal recovery, we apply the selective nature of sparse representation to perform the above mentioned classification. In order to find such sparse representation we implement an ℓ 1-minimization algorithm. This methodology overcomes the lack of robustness with respect to outliers. In contrast to other classification algorithms, no model selection dependency is involved. The minimization algorithm is a convex relaxation-like that has been proven to efficiently recover sparse signals. To study its performance, the proposed method is applied to six tumor gene expression datasets and numerically compared with various support vector machine methods (SVM). The numerical results show that the ℓ 1-minimization algorithm proposed performs at least comparably and often better than SVMs.

View PDFchevron_right
Sparse principal component analysis by choice of norm
Ruiyan Luo

Journal of Multivariate Analysis, 2013

Recent years have seen the developments of several methods for sparse principal component analysis due to its importance in the analysis of high dimensional data. Despite the demonstration of their usefulness in practical applications, they are limited in terms of lack of orthogonality in the loadings (coefficients) of different principal components, the existence of correlation in the principal components, the expensive computation needed, and the lack of theoretical results such as consistency in high-dimensional situations. In this paper, we propose a new sparse principal component analysis method by introducing a new norm to replace the usual norm in traditional eigenvalue problems, and propose an efficient iterative algorithm to solve the optimization problems. With this method, we can efficiently obtain uncorrelated principal components or orthogonal loadings, and achieve the goal of explaining a high percentage of variations with sparse linear combinations. Due to the strict convexity of the new norm, we can prove the convergence of the iterative method and provide the detailed characterization of the limits. We also prove that the obtained principal component is consistent for a single component model in high dimensional situations. As illustration, we apply this method to real gene expression data with competitive results.

View PDFchevron_right
Sparse representation approaches for the classification of high-dimensional biological data
Alioune Ngom

BMC Systems Biology, 2013

Background: High-throughput genomic and proteomic data have important applications in medicine including prevention, diagnosis, treatment, and prognosis of diseases, and molecular biology, for example pathway identification. Many of such applications can be formulated to classification and dimension reduction problems in machine learning. There are computationally challenging issues with regards to accurately classifying such data, and which due to dimensionality, noise and redundancy, to name a few. The principle of sparse representation has been applied to analyzing high-dimensional biological data within the frameworks of clustering, classification, and dimension reduction approaches. However, the existing sparse representation methods are inefficient. The kernel extensions are not well addressed either. Moreover, the sparse representation techniques have not been comprehensively studied yet in bioinformatics. Results: In this paper, a Bayesian treatment is presented on sparse representations. Various sparse coding and dictionary learning models are discussed. We propose fast parallel active-set optimization algorithm for each model. Kernel versions are devised based on their dimension-free property. These models are applied for classifying high-dimensional biological data. Conclusions: In our experiment, we compared our models with other methods on both accuracy and computing time. It is shown that our models can achieve satisfactory accuracy, and their performance are very efficient.

View PDFchevron_right
Versatile Sparse Matrix Factorization and Its Applications in High-Dimensional Biological Data Analysis
Alioune Ngom

Lecture Notes in Computer Science, 2013

Non-negative matrix factorization and sparse representation models have been successfully applied in high-throughput biological data analysis. In this paper, we propose our versatile sparse matrix factorization (VSMF) model for biological data mining. We show that many well-known sparse models are specific cases of VSMF. Through tuning parameters, sparsity, smoothness, and non-negativity can be easily controlled in VSMF. Our computational experiments corroborate the advantages of VSMF.

View PDFchevron_right

Related topics

  • Computer Science
  • Medicine

    • Cited by

    Stabilized sparse ordinal regression for medical risk stratification
    Truyen Tran

    Knowledge and Information Systems, 2014

    The recent wide adoption of Electronic Medical Records (EMR) presents great opportunities and challenges for data mining. The EMR data is largely temporal, often noisy, irregular and high dimensional. This paper constructs a novel ordinal regression framework for predicting medical risk stratification from EMR. First, a conceptual view of EMR as a temporal image is constructed to extract a diverse set of features. Second, ordinal modeling is applied for predicting cumulative or progressive risk. The challenges are building a transparent predictive model that works with a large number of weakly predictive features, and at the same time, is stable against resampling variations. Our solution employs sparsity methods that are stabilized through domain-specific feature interaction networks. We introduces two indices that measure the model stability against data resampling. Feature networks are used to generate two multivariate Gaussian priors with sparse precision matrices (the Laplacian and Random Walk). We apply the framework on a large short-term suicide risk prediction problem and demonstrate that our methods outperform clinicians to a large-margin, discover suicide risk factors that conform with mental health knowledge, and produce models with enhanced stability.

    View PDFchevron_right
    Discovery of Intermediary Genes between Pathways Using Sparse Regression
    Ashwini Patil

    PloS one, 2015

    The use of pathways and gene interaction networks for the analysis of differential expression experiments has allowed us to highlight the differences in gene expression profiles between samples in a systems biology perspective. The usefulness and accuracy of pathway analysis critically depend on our understanding of how genes interact with one another. That knowledge is continuously improving due to advances in next generation sequencing technologies and in computational methods. While most approaches treat each of them as independent entities, pathways actually coordinate to perform essential functions in a cell. In this work, we propose a methodology based on a sparse regression approach to find genes that act as intermediary to and interact with two pathways. We model each gene in a pathway using a set of predictor genes, and a connection is formed between the pathway gene and a predictor gene if the sparse regression coefficient corresponding to the predictor gene is non-zero. A...

    View PDFchevron_right
    Robust principal component analysis via capped norms
    Jieping Ye

    Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, 2013

    In many applications such as image and video processing, the data matrix often possesses simultaneously a low-rank structure capturing the global information and a sparse component capturing the local information. How to accurately extract the low-rank and sparse components is a major challenge. Robust Principal Component Analysis (RPCA) is a general framework to extract such structures. It is well studied that under certain assumptions, convex optimization using the trace norm and ℓ1-norm can be an effective computation surrogate of the difficult RPCA problem. However, such convex formulation is based on a strong assumption which may not hold in real-world applications, and the approximation error in these convex relaxations often cannot be neglected. In this paper, we present a novel nonconvex formulation for the RPCA problem using the capped trace norm and the capped ℓ1-norm. In addition, we present two algorithms to solve the non-convex optimization: one is based on the Difference of Convex functions (DC) framework and the other attempts to solve the sub-problems via a greedy approach. Our empirical evaluations on synthetic and real-world data show that both of the proposed algorithms achieve higher accuracy than existing convex formulations. Furthermore, between the two proposed algorithms, the greedy algorithm is more efficient than the DC programming, while they achieve comparable accuracy.

    View PDFchevron_right
    Structural Graphical Lasso for Learning Mouse Brain Connectivity
    Jieping Ye

    Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015

    Investigations into brain connectivity aim to recover networks of brain regions connected by anatomical tracts or by functional associations. The inference of brain networks has recently attracted much interest due to the increasing availability of high-resolution brain imaging data. Sparse inverse covariance estimation with lasso and group lasso penalty has been demonstrated to be a powerful approach to discover brain networks. Motivated by the hierarchical structure of the brain networks, we consider the problem of estimating a graphical model with tree-structural regularization in this paper. The regularization encourages the graphical model to exhibit a brain-like structure. Specifically, in this hierarchical structure, hundreds of thousands of voxels serve as the leaf nodes of the tree. A node in the intermediate layer represents a region formed by voxels in the subtree rooted at that node. The whole brain is considered as the root of the tree. We propose to apply the tree-structural regularized graphical model to estimate the mouse brain network. However, the dimensionality of whole-brain data, usually on the order of hundreds of thousands, poses significant computational challenges. Efficient algorithms that are capable of estimating networks from high-dimensional data are highly desired. To address the computational challenge, we develop a screening rule which can quickly identify many zero blocks in the estimated graphical model, thereby dramatically reducing the computational cost of solving the proposed model. It is based on a novel insight on the relationship between screening and the so-called proximal operator that we first establish in this paper. We perform experiments on both synthetic data and real data from the Allen Developing Mouse Brain Atlas; results demonstrate the effectiveness and efficiency of the proposed approach.

    View PDFchevron_right
    Gene knockout inference with variational graph autoencoder learning single-cell gene regulatory networks
    Robert Chapkin

    Nucleic Acids Research, 2023

    In this paper, we introduce Gene Knoc kout Inference (GenKI), a virtual knockout (KO) tool for gene function prediction using single-cell RNA sequencing (scRNA-seq) data in the absence of KO samples when only wild-type (WT) samples are available. Without using any information from real KO samples, GenKI is designed to capture shifting patterns in gene regulation caused by the KO perturbation in an unsupervised manner and pr o vide a r ob ust and scalable framew ork f or gene function studies. To achieve this goal, GenKI adapts a variational graph autoencoder (VGAE) model to learn latent representations of genes and interactions between genes from the input WT scRNA-seq data and a derived single-cell g ene regulator y network (scGRN). The virtual KO data is then generated by computationally removing all edges of the KO gene --the gene to be knocked out for functional study --from the scGRN. The differences between WT and virtual KO data are discerned by using their corresponding latent parameters derived from the trained VGAE model. Our simulations show that GenKI accurately approximates the perturbation profiles upon gene KO and outperforms the state-of-the-art under a series of evaluation conditions. Using publicl y a vailable scRNA-seq data sets, we demonstrate that GenKI recapitulates discoveries of real-animal KO experiments and accurately pre-dicts cell type-specific functions of KO genes. Thus, GenKI pr o vides an in-silico alternative to KO experiments that may partially replace the need for genetically modified animals or other genetically perturbed systems.

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved