Academia.eduAcademia.edu

Outline

Title
Abstract
Related Work
Background
An Overview of the Results
Representation Lemmas and Pathological Cases
Unidentifiability Results with Coding
Conclusions
References

Fundamental Limits of Blind Deconvolution Part I: Ambiguity Kernel

Profile image of Sunav ChoudharySunav Choudhary

Abstract

Blind deconvolution is an ubiquitous non-linear inverse problem in applications like wireless communications and image processing. This problem is generally ill-posed since signal identifiability is a key concern, and there have been efforts to use sparse models for regularizing blind deconvolution to promote signal identifiability. Part I of this two-part paper establishes a measure theoretically tight characterization of the ambiguity space for blind deconvolution and unidentifiability of this inverse problem under unconstrained inputs. Part II of this paper analyzes the identifiability of the canonical-sparse blind deconvolution problem and establishes surprisingly strong negative results on the sparsity-ambiguity trade-off scaling laws. Specifically, the ill-posedness of canonical-sparse blind deconvolution is quantified by exemplifying the dimension of the unidentifiable signal sets. An important conclusion of the paper is that canonical sparsity occurring naturally in applications is insufficient and that coding is necessary for signal identifiability in blind deconvolution. The methods developed herein are applied to a second-hop channel estimation problem to show that a family of subspace coded signals (including repetition coding and geometrically decaying signals) are unidentifiable under blind deconvolution.

Related papers

A general solution to blind inverse problems for sparse input signals
David Luengo

Neurocomputing, 2005

In this paper, we present a computationally efficient algorithm which provides a general solution to blind inverse problems for sparse input signals. The method takes advantage of the clustering typical of sparse input signals to identify the channel matrix, solving four problems sequentially: detecting the number of input signals (i.e. clusters), estimating the directions of the clusters, estimating their amplitudes, and ordering them. Once the channel matrix is known, the pseudoinverse can be used as the canonical solution to obtain the input signals. When the input signals are not sparse enough, the algorithm can be applied after a linear transformation of the signals into a domain where they show a good degree of sparsity. The performance of the algorithm for the different types of problems considered is evaluated using Monte Carlo simulations. r

View PDFchevron_right
Optimal sparse representations for blind source separation and blind deconvolution: a learning approach
Michael Zibulevsky

2004 International Conference on Image Processing, 2004. ICIP '04.

We present a generic approach, which allows to adapt sparse blind deconvolution and blind source separation algorithms In arbitraly sources. The key idea is to bring thc problem to the case in which the underlying sources are sparse by applying a sparsifying transformation on the mixtures. We present simulation results and show that such transfotmalion can bc found by training. Properties of the optimal sparsifying transformation are highlighted by an example with aerial images.

View PDFchevron_right
Identifiability Scaling Laws in Bilinear Inverse Problems
Sunav Choudhary

A number of ill-posed inverse problems in signal processing, like blind deconvolution, matrix factorization, dictionary learning and blind source separation share the common characteristic of being bilinear inverse problems (BIPs), i.e. the observation model is a function of two variables and conditioned on one variable being known, the observation is a linear function of the other variable. A key issue that arises for such inverse problems is that of identifiability, i.e. whether the observation is sufficient to unambiguously determine the pair of inputs that generated the observation. Identifiability is a key concern for applications like blind equalization in wireless communications and data mining in machine learning. Herein, a unifying and flexible approach to identifiability analysis for general conic prior constrained BIPs is presented, exploiting a connection to low-rank matrix recovery via 'lifting'. We develop deterministic identifiability conditions on the input signals and examine their satisfiability in practice for three classes of signal distributions, viz. dependent but uncorrelated, independent Gaussian, and independent Bernoulli. In each case, scaling laws are developed that trade-off probability of robust identifiability with the complexity of the rank two null space. An added appeal of our approach is that the rank two null space can be partly or fully characterized for many bilinear problems of interest (e.g. blind deconvolution). We present numerical experiments involving variations on the blind deconvolution problem that exploit a characterization of the rank two null space and demonstrate that the scaling laws offer good estimates of identifiability.

View PDFchevron_right
A Lifted l1/l2 Constraint for Sparse Blind Deconvolution
Felix Herrmann

Proceedings, 2015

We propose a modification to a sparsity constraint based on the ratio of 1 and 2 norms for solving blind seismic deconvolution problems in which the data consist of linear convolutions of different sparse reflectivities with the same source wavelet. We also extend the approach to the Estimation of Primaries by Sparse Inversion (EPSI) model, which includes surface related multiples. Minimizing the ratio of 1 and 2 norms has been previously shown to promote sparsity in a variety of applications including blind deconvolution. Most existing implementations are heuristic or require smoothing the 1 / 2 penalty. Lifted versions of 1 / 2 constraints have also been proposed but are challenging to implement. Inspired by the lifting approach, we propose to split the sparse signals into positive and negative components and apply an 1 / 2 constraint to the difference, thereby obtaining a constraint that is easy to implement without smoothing the 1 or 2 norms. We show that a method of multipliers implementation of the resulting model can recover source wavelets that are not necessarily minimum phase and approximately reconstruct the sparse reflectivities. Numerical experiments demonstrate robustness to the initialization as well as to noise in the data.

View PDFchevron_right
Noise sensitivity of sparse signal representations: reconstruction error bounds for the inverse problem
Brendt Wohlberg

IEEE Transactions on Signal Processing, 2003

Certain sparse signal reconstruction problems have been shown to have unique solutions when the signal is known to have an exact sparse representation. This result is extended to provide bounds on the reconstruction error when the signal has been corrupted by noise, or is not exactly sparse for some other reason. Uniqueness is found to be extremely unstable for a number of common dictionaries.

View PDFchevron_right
Sufficient conditions for regularity and strict identifiability in MIMO systems
Terrence Moore

IEEE Transactions on Signal Processing, 2004

We consider regularity and identifiability of convolutive multi-input multi-output (MIMO) systems with additive white Gaussian noise, modeling the source and finite impulse response (FIR) channel sequences as deterministic unknowns. In the blind context, the MIMO system is not locally identifiable; hence, its Fisher information matrix (FIM) is not regular. In fact, the dimension of the complex-valued blind FIM null space is at least the number of sources squared. Because the FIM is singular, additional information about the system is required to resolve the degrees of uncertainty and thereby obtain a valid Cramér-Rao bound (CRB); therefore, it is of interest to know under what conditions the blind FIM nullity reaches its lower bound. We develop sufficient conditions for the complex FIM to attain its minimum nullity, refining previous necessary conditions, and extending single-input multi-output (SIMO) results. We show that the sufficient conditions for the complex FIM to have minimum nullity are also equivalent to sufficient conditions for MIMO strict identifiability. These provide sufficient conditions on the richness of the sources, the required diversity, and the source lengths. Under these conditions, additional constraints, such as training, may be employed to yield an identifiable system with no ambiguities remaining.

View PDFchevron_right
Optimal Sparse Representations for Blind Deconvolution of Images
Michael Zibulevsky

Lecture Notes in Computer Science, 2004

The relative Newton algorithm, previously proposed for quasi maximum likelihood blind source separation and blind deconvolution of onedimensional signals is generalized for blind deconvolution of images. Smooth approximation of the absolute value is used in modelling the log probability density function, which is suitable for sparse sources. We propose a method of sparsification, which allows blind deconvolution of sources with arbitrary distribution, and show how to find optimal sparsifying transformations by training.

View PDFchevron_right
A new approach based on "soft statistics" to the nonlinear blind-deconvolution of unknown data channels
Enzo Baccarelli

IEEE Transactions on Signal Processing, 2001

View PDFchevron_right
Undercomplete Blind Subspace Deconvolution
Andras Lorincz

Journal of Machine Learning Research, 2007

We introduce the blind subspace deconvolution (BSSD) problem, which is the extension of both the blind source deconvolution (BSD) and the independent subspace analysis (ISA) tasks. We examine the case of the undercomplete BSSD (uBSSD). Applying temporal concatenation we reduce this problem to ISA. The associated `high dimensional' ISA problem can be handled by a recent technique called joint f-decorrelation

View PDFchevron_right
A Unified Framework for Identifiability Analysis in Bilinear Inverse Problems with Applications to Subspace and Sparsity Models
Yanjun Li, Yoram Bresler

Bilinear inverse problems (BIPs), the resolution of two vectors given their image under a bilinear mapping, arise in many applications. Without further constraints, BIPs are usually ill-posed. In practice, properties of natural signals are exploited to solve BIPs. For example, subspace constraints or sparsity constraints are imposed to reduce the search space. These approaches have shown some success in practice. However, there are few results on uniqueness in BIPs. For most BIPs, the fundamental question of under what condition the problem admits a unique solution, is yet to be answered. For example, blind gain and phase calibration (BGPC) is a structured bilinear inverse problem, which arises in many applications, including inverse rendering in computational relighting (albedo estimation with unknown lighting), blind phase and gain calibration in sensor array processing, and multichannel blind deconvolution (MBD). It is interesting to study the uniqueness of such problems. In this...

View PDFchevron_right

References (46)

  1. S. Choudhary and U. Mitra, "Sparse Blind Deconvolution: What Cannot Be Done," in 2014 IEEE International Symposium on Information Theory (ISIT), Honolulu, USA, Jun. 2014, pp. 3002-3006.
  2. --, "On The Impossibility of Blind Deconvolution for Geometrically Decaying Subspace Sparse Signals," in 2nd IEEE Global Conference on Signal and Information Processing (GlobalSIP), Atlanta, USA, Dec. 2014, pp. 463-467.
  3. J. Hadamard, "Sur les problèmes aux dérivées partielles et leur signification physique," Princeton University Bulletin, vol. 13, no. 28, pp. 49-52, 1902.
  4. A. N. Tikhonov and V. Y. Arsenin, Metody resheniya nekorrektnykh zadach, 3rd ed. "Nauka", Moscow, 1986.
  5. A. Ahmed, B. Recht, and J. Romberg, "Blind deconvolution using convex programming," IEEE Trans. Inf. Theory, vol. 60, no. 3, pp. 1711-1732, 2014.
  6. A. Kammoun, A. Aissa El Bey, K. Abed-Meraim, and S. Affes, "Robustness of blind subspace based techniques using p quasi-norms," in 2010 IEEE Eleventh International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), 2010, pp. 1-5.
  7. K. Herrity, R. Raich, and A. O. Hero, III, "Blind Reconstruction of Sparse Images with Unknown Point Spread Function," Computational Imaging VI, vol. 6814, no. 1, p. 68140K, 2008.
  8. D. Barchiesi and M. D. Plumbley, "Dictionary Learning of Convolved Signals," in 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2011, pp. 5812-5815.
  9. C. Hegde and R. G. Baraniuk, "Sampling and Recovery of Pulse Streams," IEEE Trans. Signal Process., vol. 59, no. 4, pp. 1505-1517, 2011.
  10. P. D. O'Grady, B. A. Pearlmutter, and S. T. Rickard, "Survey of sparse and non-sparse methods in source separation," International Journal of Imaging Systems and Technology, vol. 15, no. 1, pp. 18-33, 2005.
  11. S. Choudhary and U. Mitra, "Fundamental Limits of Blind Deconvolution Part I: Ambiguity Kernel," ArXiv e-prints, vol. abs/1411.3810, Nov. 2014. [Online]. Available: http://arxiv.org/abs/1411.3810
  12. D. L. Donoho, "Compressed Sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, 2006.
  13. K. Abed-Meraim, W. Qiu, and Y. Hua, "Blind System Identification," Proc. IEEE, vol. 85, no. 8, pp. 1310-1322, Aug. 1997.
  14. C. R. Johnson, Jr., P. Schniter, T. J. Endres, J. D. Behm, D. R. Brown, and R. A. Casas, "Blind Equalization Using the Constant Modulus Criterion: A Review," Proc. IEEE, vol. 86, no. 10, pp. 1927-1950, Oct. 1998.
  15. H. Liu, G. Xu, L. Tong, and T. Kailath, "Recent developments in blind channel equalization: From cyclostationarity to subspaces," Signal Processing, vol. 50, no. 1, pp. 83-99, 1996, special Issue on Subspace Methods, Part I: Array Signal Processing and Subspace Computations.
  16. J. H. Manton and W. D. Neumann, "Totally blind channel identification by exploiting guard intervals," Systems & Control Letters, vol. 48, no. 2, pp. 113-119, 2003.
  17. M. S. Asif, W. Mantzel, and J. K. Romberg, "Random Channel Coding and Blind Deconvolution," in 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Sep. 2009, pp. 1021-1025.
  18. S. Choudhary and U. Mitra, "On Identifiability in Bilinear Inverse Problems," in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vancouver, Canada, May 2013, pp. 4325-4329.
  19. --, "Identifiability Bounds for Bilinear Inverse Problems," in 47th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, USA, Nov. 2013, pp. 1677-1681.
  20. --, "Identifiability Scaling Laws in Bilinear Inverse Problems," ArXiv e-prints, vol. abs/1402.2637, Feb. 2014. [Online]. Available: http://arxiv.org/abs/1402.2637
  21. N. Richard and U. Mitra, "Sparse Channel Estimation for Cooperative Underwater Communications: A Structured Multichannel Approach," in 2008 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Mar. 2008, pp. 5300-5303.
  22. S. Choudhary and U. Mitra, "Sparse recovery from convolved output in underwater acoustic relay networks," in 2012 Asia-Pacific Signal Information Processing Association Annual Summit and Conference (APSIPA ASC), Hollywood, USA, Dec. 2012, pp. 1-8.
  23. N. Michelusi, U. Mitra, A. F. Molisch, and M. Zorzi, "Hybrid sparse/diffuse channels: A new model and estimators for wideband channels," in 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton), Sep. 2011, pp. 477-484.
  24. D. Barchiesi and M. D. Plumbley, "Learning Incoherent Dictionaries for Sparse Approximation Using Iterative Projections and Rotations," IEEE Trans. Signal Process., vol. 61, no. 8, pp. 2055-2065, Apr. 2013.
  25. R. Tur, Y. C. Eldar, and Z. Friedman, "Innovation rate sampling of pulse streams with application to ultrasound imaging," IEEE Trans. Signal Process., vol. 59, no. 4, pp. 1827-1842, 2011.
  26. P. Jung and P. Walk, "Sparse Model Uncertainties in Compressed Sensing with Application to Convolutions and Sporadic Communication," ArXiv e-prints, vol. abs/1404.0218, Apr. 2014. [Online]. Available: http://arxiv.org/abs/1404.0218
  27. R. Gribonval and K. Schnass, "Dictionary identification-sparse matrix-factorization via 1-minimization," IEEE Trans. Inf. Theory, vol. 56, no. 7, pp. 3523-3539, 2010.
  28. K. Jaganathan, S. Oymak, and B. Hassibi, "Sparse Phase Retrieval: Convex Algorithms and Limitations," in 2013 IEEE International Symposium on Information Theory Proceedings (ISIT), Jul. 2013, pp. 1022-1026.
  29. A. Fannjiang, "Absolute uniqueness of phase retrieval with random illumination," Inverse Problems, vol. 28, no. 7, pp. 075 008, 20, 2012.
  30. E. J. Candès, Y. C. Eldar, T. Strohmer, and V. Voroninski, "Phase retrieval via matrix completion," SIAM J. Imaging Sci., vol. 6, no. 1, pp. 199-225, 2013.
  31. E. Balas, "Projection, lifting and extended formulation in integer and combinatorial optimization," Ann. Oper. Res., vol. 140, pp. 125-161, 2005.
  32. A. Beck, "Convexity properties associated with nonconvex quadratic matrix functions and applications to quadratic programming," J. Optim. Theory Appl., vol. 142, no. 1, pp. 1-29, 2009.
  33. P. Mattila, Geometry of sets and measures in Euclidean spaces, Fractals and rectifiability, ser. Cambridge Studies in Advanced Mathematics. Cambridge University Press, Cambridge, 1995, vol. 44.
  34. E. J. Candès and B. Recht, "Exact matrix completion via convex optimization," Found. Comput. Math., vol. 9, no. 6, pp. 717-772, 2009.
  35. D. Gross, "Recovering Low-Rank Matrices From Few Coefficients in Any Basis," IEEE Trans. Inf. Theory, vol. 57, no. 3, pp. 1548-1566, 2011.
  36. G. Hollinger, S. Choudhary, P. Qarabaqi, C. Murphy, U. Mitra, G. Sukhatme, M. Stojanovic, H. Singh, and F. Hover, "Communication protocols for underwater data collection using a robotic sensor network," in 2011 IEEE GLOBECOM Workshops (GC Wkshps), Houston, USA, Dec. 2011, pp. 1308-1313.
  37. S. Choudhary, N. Kumar, S. Narayanan, and U. Mitra, "Active Target Detection with Mobile Agents," in 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, May 2014, pp. 4218-4222.
  38. G. A. Hollinger, U. Mitra, and G. S. Sukhatme, "Autonomous Data Collection from Underwater Sensor Networks using Acoustic Communication," in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep. 2011, pp. 3564-3570.
  39. L. Freitag, M. Johnson, M. Grund, S. Singh, and J. Preisig, "Integrated Acoustic Communication and Navigation for Multiple UUVs," in OCEANS, 2001. MTS/IEEE Conference and Exhibition, vol. 4, Nov. 2001, pp. 2065-2070.
  40. S. Choudhary, D. Kartik, N. Kumar, S. Narayanan, and U. Mitra, "Active Target Detection with Navigation Costs: A Randomized Benchmark," in 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton), Monticello, USA, Sep. 2014, pp. 109-115.
  41. C. R. Berger, S. Zhou, J. C. Preisig, and P. Willett, "Sparse Channel Estimation for Multicarrier Underwater Acoustic Communication: From Subspace Methods to Compressed Sensing," IEEE Trans. Signal Process., vol. 58, no. 3, pp. 1708-1721, Mar. 2010.
  42. W. Li and J. Preisig, "Estimation of Rapidly Time-Varying Sparse Channels," IEEE J. Ocean. Eng., vol. 32, no. 4, pp. 927-939, Oct. 2007.
  43. A. F. Molisch, "Ultra-Wide-Band Propagation Channels," Proc. IEEE, vol. 97, no. 2, pp. 353-371, Feb. 2009.
  44. N. Michelusi, B. Tomasi, U. Mitra, J. Preisig, and M. Zorzi, "An evaluation of the hybrid sparse/diffuse algorithm for underwater acoustic channel estimation," in OCEANS 2011, Sep. 2011, pp. 1-10.
  45. A. Molisch, D. Cassioli, C.-C. Chong, S. Emami, A. Fort, B. Kannan, J. Karedal, J. Kunisch, H. Schantz, K. Siwiak, and M. Win, "A Comprehensive Standardized Model for Ultrawideband Propagation Channels," IEEE Trans. Antennas Propag., vol. 54, no. 11, pp. 3151-3166, Nov. 2006.
  46. A. Saleh and R. Valenzuela, "A Statistical Model for Indoor Multipath Propagation," IEEE J. Sel. Areas Commun., vol. 5, no. 2, pp. 128-137, Feb. 1987.

Related papers

On the Properties of the Rank-Two Null Space of Nonsparse and Canonical-Sparse Blind Deconvolution
Sunav Choudhary

IEEE Transactions on Signal Processing, 2018

Blind deconvolution is an ubiquitous non-linear inverse problem in applications like wireless communications and image processing. This problem is generally ill-posed since signal identifiability is a key concern, and there have been efforts to use sparse models for regularizing blind deconvolution to promote signal identifiability. Part I of this two-part paper establishes a measure theoretically tight characterization of the ambiguity space for blind deconvolution and unidentifiability of this inverse problem under unconstrained inputs. Part II of this paper analyzes the identifiability of the canonical-sparse blind deconvolution problem and establishes surprisingly strong negative results on the sparsity-ambiguity trade-off scaling laws. Specifically, the ill-posedness of canonical-sparse blind deconvolution is quantified by exemplifying the dimension of the unidentifiable signal sets. An important conclusion of the paper is that canonical sparsity occurring naturally in applications is insufficient and that coding is necessary for signal identifiability in blind deconvolution. The methods developed herein are applied to a second-hop channel estimation problem to show that a family of subspace coded signals (including repetition coding and geometrically decaying signals) are unidentifiable under blind deconvolution.

View PDFchevron_right
1Fundamental Limits of Blind Deconvolution Part I: Ambiguity Kernel
Sunav Choudhary

2016

Blind deconvolution is an ubiquitous non-linear inverse problem in applications like wireless communications and image processing. This problem is generally ill-posed, and there have been efforts to use sparse models for regularizing blind deconvolution to promote signal identifiability. Part I of this two-part paper characterizes the ambiguity space of blind deconvolution and shows unidentifiability of this inverse problem for almost every pair of unconstrained input signals. The approach involves lifting the deconvolution problem to a rank one matrix recovery problem and analyzing the rank two null space of the resultant linear operator. A measure theoretically tight (parametric and recursive) representation of the key rank two null space is stated and proved. This representation is a novel foundational result for signal and code design strategies promoting identifiability under convolutive observation models. Part II of this paper analyzes the identifiability of sparsity constrai...

View PDFchevron_right
Identifiability in Blind Deconvolution with Subspace or Sparsity Constraints
Yanjun Li, Yoram Bresler

Blind deconvolution (BD), the resolution of a signal and a filter given their convolution, arises in many applications. Without further constraints, BD is ill-posed. In practice, subspace or sparsity constraints have been imposed to reduce the search space, and have shown some empirical success. However, existing theoretical analysis on uniqueness in BD is rather limited. As an effort to address the still mysterious question, we derive sufficient conditions under which two vectors can be uniquely identified from their circular convolution, subject to subspace or sparsity constraints. These sufficient conditions provide the first algebraic sample complexities for BD. We first derive a sufficient condition that applies to almost all bases or frames. For blind deconvolution of vectors in $\mathbb{C}^n$, with two subspace constraints of dimensions $m_1$ and $m_2$, the required sample complexity is $n\geq m_1m_2$. Then we impose a sub-band structure on one basis, and derive a sufficient ...

View PDFchevron_right
Blind Recovery of Sparse Signals From Subsampled Convolution
Marius Junge

IEEE Transactions on Information Theory, 2017

Subsampled blind deconvolution is the recovery of two unknown signals from samples of their convolution. To overcome the ill-posedness of this problem, solutions based on priors tailored to specific practical application have been developed. In particular, sparsity models have provided promising priors. However, in spite of the empirical success of these methods in many applications, existing analyses are rather limited in two main ways: by disparity between the theoretical assumptions on the signal and/or measurement model versus practical setups; or by failure to provide a performance guarantee for parameter values within the optimal regime defined by the information theoretic limits. In particular, it has been shown that a naive sparsity model is not a strong enough prior for identifiability in the blind deconvolution problem. Instead, in addition to sparsity, we adopt a conic constraint, which enforces spectral flatness of the signals. Under this prior together with random dictionary models, we show that the unknown sparse signals can be recovered from samples of their convolution at a rate scaling near optimally with the problem parameters. We also propose an iterative algorithm that is guaranteed to provide robust recovery at the same near optimal sample complexity provided that certain projection steps in the algorithm are successful. In our analysis, we haven't verified the success of these projection steps, but these steps are inactive with high probability. Numerical results show the empirical performance of the iterative algorithm agrees with the performance guarantee.

View PDFchevron_right
Blind Deconvolution Using Convex Programming
Ali Ahmed

IEEE Transactions on Information Theory, 2000

We consider the problem of recovering two unknown vectors, w and x, of length L from their circular convolution. We make the structural assumption that the two vectors are members of known subspaces, one with dimension N and the other with dimension K. Although the observed convolution is nonlinear in both w and x, it is linear in the rank-1 matrix formed by their outer product wx * . This observation allows us to recast the deconvolution problem as low-rank matrix recovery problem from linear measurements, whose natural convex relaxation is a nuclear norm minimization program.

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved