Regression with non compactly supported bases

Statistics & Probability Letters, 2008

We propose to address the common problem of linear estimation in linear statistical models by using a model selection approach via penalization. Depending then on the framework in which the linear statistical model is considered namely the regression framework or the inverse problem framework, a data-driven model selection criterion is obtained either under general assumptions, or under the mild assumption of model identifiability respectively. The proposed approach was stimulated by the important recent non-asymptotic model selection results due to Birg\'e and Massart mainly (Birge and Massart 2007), and our results in this paper, like theirs, are non-asymptotic and turn to be sharp. Our main contribution in this paper resides in the fact that these linear estimators are not necessarily least-squares estimators but can be any linear estimators. The proposed approach finds therefore potential applications in countless fields of engineering and applied science (image science, sig...

The Annals of Statistics, 1996

An experiment records stimulus and response for a random sample of cases. The relationship between response and stimulus is thought to be linear, the values of the slope and intercept varying by case. From such data, we construct a consistent, asymptotically normal, nonparametric estimator for the joint density of the slope and intercept. Our methodology incorporates the radial projection-slice theorem for the Radon transform, a technique for locally linear nonparametric regression and a tapered Fourier inversion. Computationally, the new density estimator is more feasible than competing nonparametric estimators, one of which is based on moments and the other on minimum distance considerations.

Summary This article introduces a new Markov chain Monte Carlo method to carry out Bayesian variable selection for very high dimensional models, when the subsets of variables with highest posterior probabilities have relatively few terms. The new method is applied to the nonparametric estimation of a regression surface, where the regression surface is rep- resented as a linear combination of a large number of basis terms. Adaptation of the sur- face estimator to local features is obtained by explicitly modeling the possibility that basis terms can be in or out of the model. This turns the nonparametric problem into a linear regression problem with the regression surface estimated by its posterior mean. The per- formance of a number of univariate and bivariate bases, including smoothing spline bases, is studied by simulation and a comparison is made with smoothing spline nonparametric estimators. We show that for the nonparametric estimation problem the new method is computationally ...

2018

We examined some robust and nonparametric procedures for a simple linear regression model when the error terms are drawn from unit normal, lognormal, Student t-10df, Cauchy, and exponential power via Monte Carlo simulation technique. The results showed that the nonparametric Theil’s method demonstrates the strongest performance gains in many cases which imply they have negligible bias and the smallest Mean Square Error (MSE). The second best results are obtained from Least Trimmed Squares (LTS) Methods with relative reductions in MSE. Though Least Absolute Deviation (LAD) and weighted Theil’s regressions gave the poorest slope estimates with negative root mean square error (RMSE) values in most cases, it is noticed that LAD is always more efficient than LSE whenever the error component is Cauchy distributed. The LSE proved to be more efficient under normality assumption where as LTS and Theil’s estimators performed better under nonnormal data conditions. (

2021

In the first part of this work, we develop a novel scheme for solving nonparametric regression problems. That is the approximation of possibly low regular and noised functions from the knowledge of their approximate values given at some random points. Our proposed scheme is based on the use of the pseudo-inverse of a random projection matrix, combined with some specific properties of the Jacobi polynomials system, as well as some properties of positive definite random matrices. This scheme has the advantages to be stable, robust, accurate and fairly fast in terms of execution time. Moreover and unlike most of the existing nonparametric regression estimators, no extra regularization step is required by our proposed estimator. Although, this estimator is initially designed to work with random sampling set of uni-variate i.i.d. random variables following a Beta distribution, we show that it is still work for a wide range of sampling distribution laws. Moreover, we briefly describe how ...

Journal of Statistical Planning and Inference, 1997

First, we show that many robust estimates of regression which depend only on the regression residuals (including M-, S-, Tau-, least median of squares-, least trimmed of squares-and some R-estimates) have infinite gross-error-sensitivity. More precisely, we show that the maximumbias function of a large class of estimates, called residual admissible in Yohai and Zamar (Ann. Statist. :21, 1993, 1824-1842, is of order v/~ near zero. Based on this finding we define a new robustness measure for estimates with Br(~)= o(~/~), the contamination sensitivity of order /3, which extends Hampel's gross error sensitivity for estimates with unbounded influence. We compute this measure for regression M-estimates with a general scale and show that ]~ = 0.5 in this case. Then we solve a Hampel-like optimality problem, namely, one of minimizing the asymptotic variance subject to a bound on the contamination sensitivity of order fl = 0.5, for estimates in this class. Finally, we show that a certain least ~-quantile estimate has the smallest contamination sensitivity of order 0.5 among all residual admissible estimates. In the Gaussian case c~ =0.683. (~) 1997 Elsevier Science B.V.

In this paper, we consider the problem of estimating a regression function when the outcome is censored. Two strategies of estimation are proposed: a two-step strategy where the ratio of two projection estimators is used to estimate the regression function; a direct strategy based on a standard mean-square contrast for censored data. For both estimators, non-asymptotic bounds for the integrated mean-square risk are provided and data-driven model selection is performed. In most cases, asymptotically optimal minimax rates of convergence are obtained, when the regression function belongs to a class of Besov functions.

2004

We propose a new robust estimator of the regression function in a nonparametric regression model. Empirical evidence suggests that the proposed estimator performs better (in an averaged squared error sense) than the popular local least squares estimator for non normal error distributions. The proposed estimator has the same asymptotic properties as all classical estimators.

arXiv: Statistics Theory, 2020

In the first part of this work, we develop and study a random pseudo-inverse based scheme for a stable and robust solution of nonparametric regression problems. For the interval $I=[-1,1],$ we use a random projection over a special orthonormal family of a weighted $L^2(I)-$space, given by the Jacobi normalized polynomials. Then, the pseudo-inverse of this random matrix is used to compute the different expansion coefficients of the nonparametric regression estimator with respect to this orthonormal system. We show that this estimator is stable. Then, we combine the RANdom SAmpling Consensus (RANSAC) iterative algorithm with the previous scheme to get a robust and stable nonparametric regression estimator. This estimator has also the advantage to provide fairly accurate approximations to the true regression functions. In the second part of this work, we extend the random pseudo-inverse scheme technique to build a stable and accurate estimator for solving linear functional regression (...