Reducing bias in curve estimation by use of weights

2004

In the case of one dimensional kernel density estimation on independent identically distributed observations, we suggest a cross-validation technique for the selection of a kernel from the class of all $L_2$-integrable kernel functions which have a unimodal Fourier transform. The resulting estimator is both asymptotically MISE-efficient and asymptotically minimax-adaptive over the whole scale of Sobolev classes with smoothness index greater than 1/2, including $infty$. The proofs rely on an additive decomposition of the empirical processes involved. The technique is further applied to non-parametric regression estimation. The minimax risk is established on Sobolev classes with non-integer smoothness index.

Statistics & Probability Letters, 2014

For order q kernel density estimators we show that the constant b q in bias = b q h q + o(h q) can be made arbitrarily small, while keeping the variance bounded. A data-based selection of b q is presented and Monte Carlo simulations illustrate the advantages of the method.

AStA Advances in Statistical Analysis, 2013

On the one hand, kernel density estimation has become a common tool for empirical studies in any research area. This goes hand in hand with the fact that this kind of estimator is now provided by many software packages. On the other hand, since about three decades the discussion on bandwidth selection has been going on. Although a good part of the discussion is about nonparametric regression, this parameter choice is by no means less problematic for density estimation. This becomes obvious when reading empirical studies in which practitioners have made use of kernel densities. New contributions typically provide simulations only to show that the own selector outperforms some of the existing methods. We review existing methods and compare them on a set of designs that exhibit few bumps and exponentially falling tails. We concentrate on small and moderate sample sizes because for large ones the differences between consistent methods are often negligible, at least for practitioners. As a byproduct we find that a mixture of simple plug-in and cross-validation methods produces bandwidths with a quite stable performance.

Sankhya A

Estimator selection has become a crucial issue in non parametric estimation. Two widely used methods are penalized empirical risk minimization (such as penalized log-likelihood estimation) or pairwise comparison (such as Lepski's method). Our aim in this paper is twofold. First we explain some general ideas about the calibration issue of estimator selection methods. We review some known results, putting the emphasis on the concept of minimal penalty which is helpful to design data-driven selection criteria. Secondly we present a new method for bandwidth selection within the framework of kernel density density estimation which is in some sense intermediate between these two main methods mentioned above. We provide some theoretical results which lead to some fully data-driven selection strategy.

The Stata Journal: Promoting communications on statistics and Stata, 2003

Variable bandwidth kernel density estimators increase the window width at low densities and decrease it where data concentrate. This represents an improvement over the fixed bandwidth kernel density estimators. In this article, we explore the use of one implementation of a variable kernel estimator in conjunction with several rules and procedures for bandwidth selection applied to several real datasets. The considered examples permit us to state that when working with tens or a few hundreds of data observations, least-squares cross-validation bandwidth rarely produces useful estimates; with thousands of observations, this problem can be surpassed. Optimal bandwidth and biased cross-validation (BCV), in general, oversmooth multimodal densities. The Sheather–Jones plug-in rule pro-duced bandwidths that behave slightly better in this respect. The Silverman test is considered as a very sophisticated and safe procedure to estimate the number of modes in univariate distributions; however,...

MTISD 2008. Methods, Models …, 2008

Nonparametric kernel density estimation method makes no assumptions on the functional form of the curves of interest and hence allows flexible modeling of the data. Many authors pointed out that the crucial problem in kernel density estimation method is how to determine the bandwidth (smoothing) parameter. In this paper, we introduce the most important bandwidth selection methods. In particular, least squares cross-validation, biased cross-validation, direct plug-in, solvethe-equation rules and contrast methods are considered. These methods are described and their expressions are presented. Our main practical contribution is a comparative simulation study that aims to isolate the most promising methods. The performance of each method is evaluated on the basis of the mean integrated squared error and for small-to-moderate sample size. The simulation results showed that the contrast method is the most promising methods based on the simulated families that are considered.

Canadian Journal of Statistics-revue Canadienne De Statistique, 2009

There are several levels of sophistication when specifying the bandwidth matrix H to be used in a multivariate kernel density estimator, including H to be a positive multiple of the identity matrix, a diagonal matrix with positive elements or, in its most general form, a symmetric positive-definite matrix. In this paper, the author proposes a data-based method for choosing the smoothing parametrization to be used in the kernel density estimator. The procedure is fully illustrated by a simulation study and some real data examples. The Canadian Journal of Statistics © 2009 Statistical Society of CanadaIl y a plusieurs niveaux de raffinement possible lorsque nous devons spécifier la matrice de la largeur de fenêtre, H, à utiliser dans un estimateur par le noyau d'une densité multidimensionnelle. Les choix possibles pour H sont : un multiple positif de la matrice identité, une matrice diagonale avec des entrées positives ou, dans la forme plus générale, une matrice symétrique définie positive. Dans cet article, nous proposons une méthode adaptative pour choisir la paramétrisation de lissage à utiliser dans l'estimateur par le noyau. La méthode est illustrée à l'aide d'une étude de simulation et quelques vrais jeux de données. La revue canadienne de statistique © 2009 Société statistique du Canada

Annals of Statistics, 2000

We consider methods for kernel regression when the explanatory and/or response variables are adjusted prior to substitution into a conventional estimator. This "data-sharpening" procedure is designed to preserve the advantages of relatively simple, low-order techniques, for example, their robustness against design sparsity problems, yet attain the sorts of bias reductions that are commonly associated only with high-order methods. We consider Nadaraya-Watson and local-linear methods in detail, although data sharpening is applicable more widely. One approach in particular is found to give excellent performance. It involves adjusting both the explanatory and the response variables prior to substitution into a local linear estimator. The change to the explanatory variables enhances resistance of the estimator to design sparsity, by increasing the density of design points in places where the original density had been low. When combined with adjustment of the response variables, it produces a reduction in bias by an order of magnitude. Moreover, these advantages are available in multivariate settings. The data-sharpening step is simple to implement, since it is explicitly defined. It does not involve functional inversion, solution of equations or use of pilot bandwidths.

Biometrika, 2004

This paper proposes an algorithm for boosting kernel density estimates. We show that boosting is closely linked to a previously proposed method of bias reduction and indicate how it should enjoy similar properties. Numerical examples and simulations are used to illustrate the findings, and we also suggest further areas of research.

Statistics and Computing, 2000

A new procedure is proposed for deriving variable bandwidths in univariate kernel density estimation, based upon likelihood cross-validation and an analysis of a Bayesian graphical model. The procedure admits bandwidth selection which is flexible in terms of the amount of smoothing required. In addition, the basic model can be extended to incorporate local smoothing of the density estimate. The method is shown to perform well in both theoretical and practical situations, and we compare our method with those of Abramson (The Annals of Statistics 10: 1217-1223) and Sain and Scott (Journal of the American Statistical Association 91: 1525-1534). In particular, we note that in certain cases, the Sain and Scott method performs poorly even with relatively large sample sizes. We compare various bandwidth selection methods using standard mean integrated square error criteria to assess the quality of the density estimates. We study situations where the underlying density is assumed both known and unknown, and note that in practice, our method performs well when sample sizes are small. In addition, we also apply the methods to real data, and again we believe our methods perform at least as well as existing methods.