Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Numerical Results and Conclusions
References
All Topics
Mathematics
Statistics

Bias robustness of three median-based regression estimates

Profile image of Ruben ZamarRuben Zamar

2004, Journal of Statistical Planning and Inference

Abstract

The need for regression estimates with small biases has been highlighted in view of the renewed interest in robust inference beyond point estimation. Estimates with small biases are essential for stable and informative robust inference (see Outliers in Statistical Data by V. T. Lewis, 1994, p. 74 and Fraiman et al., 2001, Ann. Statist, to appear). In this paper we study the bias performance of three median-based estimates: Brown and Mood (1951) estimate, and Sen (1968) estimate and Siegel's repeated median (1982), which exhibits an outstanding bias performance. We also consider a one-step version of Brown and Mood's estimate. We pay special attention to the maximum asymptotic bias of the intercept parameter which has been mostly ignored in the robustness literature (with exceptions pointed out in the introduction).

Related papers

On Bias Reduction in Robust Inference for Generalized Linear Models
Brajendra Sutradhar

Scandinavian Journal of Statistics, 2010

It is well known that one or more outlying points in the data may adversely affect the consistency of the quasi-likelihood or the likelihood estimators for the regression effects. Similar to the quasi-likelihood approach, the existing outliers-resistant Mallow's type quasi-likelihood (MQL) estimation approach may also produce biased regression estimators. As a remedy, by using a fully standardized score function in the MQL estimating equation, in this paper, we demonstrate that the fully standardized MQL estimators are almost unbiased ensuring its higher consistency performance. Both count and binary responses subject to one or more outliers are used in the study. The small sample as well as asymptotic results for the competitive estimators are discussed.

View PDFchevron_right
Robust median estimator in logistic regression
Leandro Pardo

Journal of Statistical Planning and Inference, 2008

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier's archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright

View PDFchevron_right
Min-Max Bias Robust Regression
Ruben Zamar

The Annals of Statistics, 1989

This the problem maximum asymptotic bias of regression estimates over a-contamination for the joint of the response carriers. Two classes of estimates are treated: (1) Msestimates with bounded function p applied to the scaled residuals, using a very general class of scale estimates, and (2) Bounded influence function type generalized M-estimates. Estimates in the first class are oblta1rled as problem, while estimates in the second class are specified by an estimating equation. The first class of M-estimates is sufficiently general to include both Huber "Proposal 2" simultaneous estimates of regression coefficients and residuals scale, and Rousseeuw-Yohai "Sestimates" of regression [Robust and Nonlinear Time Series (1984): 256-272]. It is shown than an S-estimate based on a jump-function type p solves the min-max bias problem for the class of M-estimates with very general scale. This .estimate i~obtained by the. nrinimizationof the a-quantile of the squared residuals, where a = a(e)· depends on the fraction of contamination e. When e~.5, cc(e)~,5 and the min-max estimator approaches least median of residuals estimator introduced by Rcusseeuwj/, Am. Statist. Assoc., 79]. For the bounded influence class of

View PDFchevron_right
“Hodges-Lehmann Sign-based Estimators and Generalized Confidence Distributions in Linear Median Regressions with Moment-free Heterogenous Errors and Dependence of Unknown Form”
Jean-Marie Dufour

This paper develops sign-based estimation methods for the parameters of a median regression in finite samples. We introduce p-value functions that give the confidence one may have in a certain value of the parameter given the realization of the sample and sign-based estimators that are the values associated with the highest confidence (p-value). The sign-based estimators are thus obtained using the Hodges-Lehmann principle of test inversion. They are expected to present the same robustness properties than the test statistics they come from and can straightly be associated with the finite-sample-based inference procedure described in . We also show they are median unbiased (under symmetry and estimator unicity) and present equivariance features similar to the LAD estimator. Consistency under point identification and asymptotic normality are provided and hold under weaker assumptions than the LAD estimator. However, small sample behavior is our first interest. By a Monte Carlo study of bias and RMSE, we show sign-based estimators perform better than the LAD in very heteroskedastic settings.

View PDFchevron_right
Appendix F: Robust Statistics
Bala Arshanapalli

2014

View PDFchevron_right
Empirical Robust Multivariate Regression Parameter Estimation Using Median Approach
ok sajana

International Journal of Scientific Research in Mathematical and Statistical Sciences

Main purpose of multivariate regression analysis is the estimation of model parameters. The use of maximum likelihood method would not be appropriate in estimation problems while data contains outlier or extreme observations. So it is necessary to find a parameter estimation method in which the value of the estimator is not much affected by small changes in the data. This paper introduces robust method for multivariate regression based on robust estimation of location and scatter matrix of predictor and response variables. In this paper Comedian method is taken as a robust estimator of location and scatter. Based on the simulations, the finite-sample efficiency and robustness of the estimator are investigated. Efficiency of proposed robust estimators is compared with maximum likelihood estimator, minimum covariance determinant estimator and orthogonalized Gnanadesikan-Kettenring estimator in terms of mean squared errors. Proposed estimator combines high robustness and high efficiency in estimation. The proposed method is illustrated on a real data set.

View PDFchevron_right
IMPROVED ESTIMATION FOR ROBUST ECONOMETRIC REGRESSION MODELS
Klaus Vasconcellos, Gauss Cordeiro

2000

The t distribution has proved to be a useful alternative to the normal distribution in many econometric regression models, especially when robust estimation is desired. In this work, we consider a nonlinear heteroskedastic Student t regression model. We suppose the observations to be independently t distributed, with the location and scale parameters for each observation being related to linear combinations of some explanatory variables, through regular, and possibly nonlinear, completely known link functions. We obtain the second order biases of the maximum likelihood estimates of the coefficients of those linear combinations and show that the biases will only depend on the first two derivatives of the link functions. We also express the biases in a closed matrix form, allowing them to be easily computed, in practical applications, from auxiliary generalized linear regressions. We discuss some important special cases and present Monte Carlo simulation results indicating that the bias-corrected estimates outperform the corresponding uncorrected estimates for relatively small sample sizes. An example with real data showing the usefulness of bias correction for this model is also presented.

View PDFchevron_right
Robust Regression Using Multiple Repeated Median of Slope
Bamidele Oseni

African Journal of Applied Statistics, 2017

A generalization of the Repeated Median of Slope (RMS) is carried out to accommodate multiple regression models. This is obtained through the investigation of the behavior of total change in the dependent variable with respect to an independent variable. The asymptotic behavior of the estimator is investigated when certain percentage of the observations come from contamination-outlier generating unknown distribution. The performance of the estimator is compared with that of the ordinary least square (OLS) and huber estimator using bias, variance and RMSE. Expectedly, the estimator is consistent and more efficient than the OLS and huber when the observations are contaminated with outlier. Though using RMSE as evaluation criteria its performance is comparably very poor.

View PDFchevron_right
Leverages, Outliers and the Performance of Robust Regression Estimators
SIMON APPIAH

British Journal of Mathematics & Computer Science, 2016

In this study, we assess the performance of some robust regression methods. These are the least-trimmed squares estimator (LTSE), Huber maximum likelihood estimator (HME), S-Estimator (SE) and modified maximum likelihood estimator (MME) which are compared with the ordinary least squares Estimator (OLSE) at different levels of leverages in the predictor variables. Anthropometric data from Komfo Anokye Teaching Hospital (KATH) was used and the comparison is done using root mean square error (RMSE), relative efficiencies (RE), coefficients of determination (R-squared) and power of the test. The results show that robust methods are as efficient as the OLSE if the assumptions of OLSE are met. OLSE is affected by low and high percentage of leverages, HME broke-down with leverages in data. MME and SE are robust to all percentage of aberrations, while LTSE is slightly affected by high percentage leverages perturbation. Thus, MME and SE are the most robust methods, while OLSE and HME are the least robust and the performance of the LTSE is affected by higher percentages of leverage in this study.

View PDFchevron_right
Median estimation through a regression transformation
Zhenlin Yang

Canadian Journal of Statistics, 2002

The author considers the problem of constructing confidence intervals for the median of a future observation at certain values of exogenous variables, following a normalizing transformation. He shows that when this transformation is estimated, the usual interval obtained through an inverse transformation needs to be corrected, even when the sample size is large. He then gives a simple analytical solution to this problem and provides simulation results confirming the good small-sample properties of the corrected interval. He also presents two concrete illustrations. L'estimation de la médiane par l'intermédiaire d'une transformation de la régression Résumé : L'auteur s'intéresseà la construction d'intervalles de confiance pour la médiane d'une observation future correspondantà certaines valeurs de variables exogènes, suiteà une transformation normalisatrice. Il montre que lorsque cette transformation est estimée, l'intervalle usuel obtenu par transformation inverse doitêtre corrigé, même pour de grandes tailles d'échantillon. Il propose ensuite une solution analytique simpleà ce problème et rapporte des résultats de simulation attestant des bonnes propriétés de l'intervalle corrigé dans de petitséchantillons. Il présente en outre deux illustrations pratiques.

View PDFchevron_right

References (43)

  1. References Adrover, J.G. (1998). Minimax bias robust estimation of the dispersion matrix of a multivariate distribution. Ann. Statist, 26, 2301-2320.
  2. Adrover, J. G. and Zamar, R. H. (2000). Bias robustness of three median-based regression esti- mates. Tech. Rep. 194 , Department of Statistics, UBC, Vancouver, Canada.
  3. Barnett, V. and Lewis, T. (1994). Outliers in statistical data. Wiley & Sons. New York.
  4. Berrendero, J.R. and Romo, J.J (1998). Stability under contamination of robust regression esti- mates based on differences of residuals. J. Statist. Plann. Infer., 70, 149-165.
  5. Berrendero, J.R. and Zamar, R.H. (2001). Maximum bias curves for robust regression with non- elliptical regressors. Ann. Statist., 29.
  6. Brown, G.W. and Mood, A. M. (1951). On median tests for linear hypotheses, in Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, Univ. of California Press, Berkeley, 159-166.
  7. Coakley, C. and Hettmansperger, T.P. (1993). A bounded influence, high breakdown, efficient regression estimator. J. Amer. Statist. Assoc., 88, 872-880.
  8. Croux, C., Rousseeuw, P.J. and Hössjer, O. (1994). Generalized S-estimators, J. Amer. Statist. Assoc., 79, 1271-1281.
  9. Davies, P.L. (1993). Aspects of robust linear regression. Ann. Statist, 21, 1843-1899.
  10. Donoho, D.L. and Liu, R.C. (1988). The "automatic" robustness of minimum distance functionals. Ann. Statist, 16, 552-586.
  11. Fraiman, R., Yohai, V.J. and Zamar (2001). Optimal robust M-estimates of location. Ann. Statist, 29.
  12. Hampel, F.R. (1971). General qualitative definition of robustness. Ann. Math. Statist, 42, 1887-1896.
  13. Hampel, F.R. (1974). The influence curve and its role in robust estimation, J. Amer. Statist. Assoc., 69, 383-393.
  14. Hampel, F.R., Ronchetti, E. Z., Rousseeuw, P.J. and Stahel, W. A. (1986). Robust Statistics. The approach based on influence functions. Wiley Series in Probability and Mathematical Statistics, New York.
  15. He, X. (1991). A local breakdown point property of robust tests in linear regression. J. Multivariate Anal., 38, 294-305.
  16. He, X., Simpson, D.G. and Portnoy, S.L. (1990). Breakdown robustness of tests. J. Amer. Statist. Assoc., 85, 446-452.
  17. He, X. and Simpson D.G. (1993). Lower bound for contamination bias: globally minimax versus locally linear estimation. Ann. Statist., 21, 314-337.
  18. Hennig, Christian (1995). Efficient high-breakdown point estimators in robust regression: Which function to choose? Statistics & Decisions, 13, 221-241.
  19. Hössjer, O, Rousseeuw, P.J. and Croux, C. (1994). Asymptotics of the repeated median slope estimator, Ann. Statist, 22, 1478-1501.
  20. Huber, P.J. (1964). Robust estimation of a location parameter, Ann. Math. Statist., 35, 73-101.
  21. Huber, P.J. (1981). Robust Statistics. Wiley, New York.
  22. Maronna, R.A., Bustos, O.H. and Yohai, V.J. (1979). Bias-and-efficiency-robustness of general M-estimators for regression with random carriers. In Smoothing Techniques for Curve Esti- mation, T. Gasser and M. Rosemblatt (eds.). Lecture Notes in Mathematics 757. Springer, Berlin, 91-116.
  23. Martin, R.D. and Zamar, R.H. (1989). Asymptotically min-max bias robust M-estimates of scale for positive random variables. J. Amer. Statist. Assoc., 84, 494-501.
  24. Martin, R.D. and Zamar, R.H. (1993). Bias robust estimation of scale. Ann. Statist., 21, 991- 1017.
  25. Martin, R. D., Yohai, V. J. and Zamar, R. H. (1989). Min-max bias robust regression. Ann. Statist., 17, 1608-1630.
  26. Riedel, M. (1987). The asymptotic bias in the deviation of the local model. Stability Problems for Stochastic Models. Lecture Notes in Math., 1233, 134-144. Springer, Berlin.
  27. Riedel, M. (1989a). On the bias-robustness in the location model I. Statistics, 20, 223-234.
  28. Riedel, M. (1989b). On the bias-robustness in the location model II. Statistics, 20, 235-246.
  29. Rousseeuw, P.J. (1984). Least median of squares regression. J. Amer. Statist. Assoc., 79, 871-880.
  30. Rychlik, T. (1987). An asymptotically most bias-stable estimator of location. Statistics, 18, 563-571.
  31. Rychlik, T. and Zielinski, R. (1987). An asymptotically most bias-robust invariant estimator of location. Stability Problems for Stochastic Models. Lecture Notes in Math., 1233, 156-171. Springer, Berlin.
  32. Sen, P.K. (1968). Estimates of the regression coefficient based on Kendall's Tau. J. Amer. Statist. Assoc., 63, 1379-1389.
  33. Siegel, A. (1982). Robust regression using repeated medians. Biometrika, 69, 242-244.
  34. Simpson, D. G., Ruppert, and Carroll, R. J. (1992). On one-step GM-estimates and stability of inferences in linear regression. J. Amer. Statist. Assoc., 87, 439-450.
  35. Theil, H. (1950). A rank-invariant method of linear and polynomial regression analysis, I, II, and III. In Nederl. Akad. Wetensch. Proc., pp. 386-392, 521-525, and 1397-1412.
  36. Yohai, V. J. (1987). High-breakdown point and high efficiency robust estimates for regression. Ann. Statist., 15, 642-656.
  37. Yohai, V.J. and Zamar, R. H. (1988). High breakdown point estimates of regression by means of the minimization of an efficient scale. J. Amer. Statist. Assoc., 83, 406-413.
  38. Yohai, V.J. and Zamar, R. H. (1993). A minimax-bias property of the least α-quantile estimates. Ann. Statist. 21, 1824-1842.
  39. Yohai, V.J. and Zamar, R. H. (1997). Optimal locally robust M-estimates of regression. J. Statist. Plann. Inference, 64, 309-323.
  40. Zamar, R.H. (1992). Bias robust estimation in orthogonal regression. Ann. Statist. 20, 1875-1888.
  41. Zielinski, R. (1985). A most bias-robust estimate of the location parameter: a general existence theorem. Statistics, 16, 637-640.
  42. Zielinski, R. (1987). Robustness of sample mean and median under restrictions on outliers. Zastos. Mat.,19, 239-240.
  43. Zielinski, R. (1988). A distribution-free median-unbiased quantile estimator. Statistics, 19, 223- 227.

Related papers

Finite-sample generalized confidence distributions and sign-based robust estimators in median regressions with heterogeneous dependent errors
Jean-Marie Dufour

We study the problem of estimating the parameters of a linear median regression without any assumption on the shape of the error distribution -- including no condition on the existence of moments -- allowing for heterogeneity (or heteroskedasticity) of unknown form, noncontinuous distributions, and very general serial dependence (linear and nonlinear). This is done through a reverse inference approach, based on a distribution-free testing theory [Coudin and Dufour (2009, The Econometrics Journal)], from which confidence sets and point estimators are subsequently generated. The estimation problem is tackled in two complementary ways. First, we show how confidence distributions for model parameters can be applied in such a context. Such distributions -- which can be interpreted as a form of fiducial inference -- provide a frequency-based method for associating probabilities with subsets of the parameter space (like posterior distributions do in a Bayesian setup) without the introduction of prior distributions. We consider generalized confidence distributions applicable to multidimensional parameters, and we suggest the use of a projection technique for confidence inference on individual model parameters. Second, we propose point estimators, which have a natural association with confidence distributions. These estimators are based on maximizing test p-values and inherit robustness properties from the generating distribution-free tests. Both finite-sample and large-sample properties of the proposed estimators are established under weak regularity conditions. We show they are median unbiased (under symmetry and estimator unicity) and possess equivariance properties. Consistency and asymptotic normality are established without any moment existence assumption on the errors, allowing for noncontinuous distributions, heterogeneity and serial dependence of unknown form. These conditions are considerably weaker than those used to show corresponding results for LAD estimators. In a Monte Carlo study of bias and RMSE, we show sign-based estimators perform better than LAD-type estimators in heteroskedastic settings. We present two empirical applications, which involve financial and macroeconomic data, both affected by heavy tails (non-normality) and heteroskedasticity: a trend model for the S&P index, and an equation used to study β-convergence of output levels across U.S. States.

View PDFchevron_right
Estimation of Population Median under Robust Measures of an Auxiliary Variable
Zohaib Sarwar

Mathematical Problems in Engineering, 2021

In this paper, a generalized class of estimators for the estimation of population median are proposed under simple random sampling without replacement (SRSWOR) through robust measures of the auxiliary variable. Three robust measures, decile mean, Hodges–Lehmann estimator, and trimean of an auxiliary variable, are used. Mathematical properties of the proposed estimators such as bias, mean squared error (MSE), and minimum MSE are derived up to first order of approximation. We considered various real-life datasets and a simulation study to check the potentiality of the proposed estimators over the competitors. Robustness is also examined through a real dataset. Based on the fascinating results, the researchers are encouraged to use the proposed estimators for population median under SRSWOR.

View PDFchevron_right
Robust Sign-Based and Hodges-Lehmann Estimators in Linear Median Regressions with Heterogenous Serially Dependent Errors
Jean-Marie Dufour

SSRN Electronic Journal, 2000

View PDFchevron_right
A new estimator Median of the distribution of the mean in robustness
Alfonso García-Pérez

In some statistical methods, the statistical information is provided in terms of the values used by classical estimators, such as the sample mean and sample variance. These estimations are used in a second stage, usually in a classical manner, to be combined into a single value, as a weighted mean. Moreover, in many applied studies, the results are given in these terms, i.e., as summary data. In all of these cases, the individual observations are unknown; therefore, computing the usual robustness estimators with them to replace classical non-robust estimations by robust ones is not possible. In this paper, the use of the median of the distribution Fx of the sample mean is proposed, assuming a locationscale contaminated normal model, where the parameters of Fx are estimated with the classical estimations provided in the first stage. The estimator so defined is called median of the distribution of the mean, MdM. This new estimator is applied in Mendelian randomization, defining the new robust inverse weighted estimator, RIVW.

View PDFchevron_right
Robust nonparametric inference for the median
Ruben Zamar

Eprint Arxiv Math 0503665, 2005

We consider the problem of constructing robust nonparametric confidence intervals and tests of hypothesis for the median when the data distribution is unknown and the data may contain a small fraction of contamination. We propose a modification of the sign test (and its associated confidence interval) which attains the nominal significance level (probability coverage) for any distribution in the contamination neighborhood of a continuous distribution. We also define some measures of robustness and efficiency under contamination for confidence intervals and tests. These measures are computed for the proposed procedures.

View PDFchevron_right

Related topics

  • Statistics
  • Statistical Regression
    • 580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved