Optimal Experimental Designs for Multilevel Logistic Models

Statistics in Medicine, 2000

The use of multi-level logistic regression models was explored for the analysis of data from a cluster randomized trial investigating whether a training programme for general practitioners' reception sta! could improve women's attendance at breast screening. Twenty-six general practices were randomized with women nested within them, requiring a two-level model which allowed for between-practice variability. Comparisons were made with "xed e!ect (FE) and random e!ects (RE) cluster summary statistic methods, ordinary logistic regression and a marginal model based on generalized estimating equations with robust variance estimates. An FE summary statistic method and ordinary logistic regression considerably understated the variance of the intervention e!ect, thus overstating its statistical signi"cance. The marginal model produced a higher statistical signi"cance for the intervention e!ect compared to that obtained from the RE summary statistic method and the multi-level model. Because there was only a moderate number of practices and these had unbalanced cluster sizes, reliable asymptotic properties for the robust standard errors used in the marginal model may not have been achieved. While the RE summary statistic method cannot handle multiple covariates easily, marginal and multi-level models can do so. In contrast to multi-level models however, marginal models do not provide direct estimates of variance components, but treat these as nuisance parameters. Estimates of the variance components were of particular interest in this example. Additionally, parametric bootstrap methods within the multi-level model framework provide con"dence intervals for these variance components, as well as a con"dence interval for the e!ect of intervention which allows for the imprecision in the estimated variance components. The assumption of normality of the random e!ects can be checked, and the models extended to investigate multiple sources of variability.

Behavior Research Methods, 2009

BMC Medical Research Methodology, 2007

Background: Many studies conducted in health and social sciences collect individual level data as outcome measures. Usually, such data have a hierarchical structure, with patients clustered within physicians, and physicians clustered within practices. Large survey data, including national surveys, have a hierarchical or clustered structure; respondents are naturally clustered in geographical units (e.g., health regions) and may be grouped into smaller units. Outcomes of interest in many fields not only reflect continuous measures, but also binary outcomes such as depression, presence or absence of a disease, and self-reported general health. In the framework of multilevel studies an important problem is calculating an adequate sample size that generates unbiased and accurate estimates.

The Annals of Statistics, 2020

We consider optimal designs for general multinomial logistic models, which cover baseline-category, cumulative, adjacent-categories, and continuation-ratio logit models, with proportional odds, non-proportional odds, or partial proportional odds assumption. We derive the corresponding Fisher information matrices in three different forms to facilitate their calculations, determine the conditions for their positive definiteness, and search for optimal designs. We conclude that, unlike the designs for binary responses, a feasible design for a multinomial logistic model may contain less experimental settings than parameters, which is of practical significance. We also conclude that even for a minimally supported design, a uniform allocation, which is typically used in practice, is not optimal in general for a multinomial logistic model. We develop efficient algorithms for searching D-optimal designs. Using examples based on real experiments, we show that the efficiency of an experiment can be significantly improved if our designs are adopted.

Psychological methods, 2017

The focus of this article is to describe Bayesian estimation, including construction of prior distributions, and to compare parameter recovery under the Bayesian framework (using weakly informative priors) and the maximum likelihood (ML) framework in the context of multilevel modeling of single-case experimental data. Bayesian estimation results were found similar to ML estimation results in terms of the treatment effect estimates, regardless of the functional form and degree of information included in the prior specification in the Bayesian framework. In terms of the variance component estimates, both the ML and Bayesian estimation procedures result in biased and less precise variance estimates when the number of participants is small (i.e., 3). By increasing the number of participants to 5 or 7, the relative bias is close to 5% and more precise estimates are obtained for all approaches, except for the inverse-Wishart prior using the identity matrix. When a more informative prior w...

PLOS ONE

Educational researchers, psychologists, social, epidemiological and medical scientists are often dealing with multilevel data. Sometimes, the response variable in multilevel data is categorical in nature and needs to be analyzed through Multilevel Logistic Regression Models. The main theme of this paper is to provide guidelines for the analysts to select an appropriate sample size while fitting multilevel logistic regression models for different threshold parameters and different estimation methods. Simulation studies have been performed to obtain optimum sample size for Penalized Quasi-likelihood (PQL) and Maximum Likelihood (ML) Methods of estimation. Our results suggest that Maximum Likelihood Method performs better than Penalized Quasi-likelihood Method and requires relatively small sample under chosen conditions. To achieve sufficient accuracy of fixed and random effects under ML method, we established ''50/50" and ''120/50" rule respectively. On the basis our findings, a ''50/60" and ''120/70" rules under PQL method of estimation have also been recommended.

Biometrics, 2015

Computational Statistics & Data Analysis, 2004

Hierarchical models are common in complex surveys, psychometric applications, as well as agricultural and biomedical applications, to name but a few. The context of interest here is meta-analysis, with emphasis on the use of such an approach in the evaluation of surrogate endpoints in randomized clinical trials. The methodology rests on the ability to replicate the e ect of treatment on both the true endpoint, as well as the candidate surrogate endpoint, across a number of trials. However, while a meta-analysis of clinical trials in the same indication seems the natural hierarchical structure, some authors have considered center or country as the unit, either because no meta-analytic data were available or because, even when available, they might not allow for a su cient level of replication. This leaves us with two important, related questions. First, how sensible is it to replace one level of replication by another one? Second, what are the consequences when a truly three-or higher-level model (e.g., trial, center, patient) is replaced by a coarser two-level structure (either trial and patient or center and patient). The same or similar questions may occur in a number of di erent settings, as soon as interest is placed on the validity of a conclusion at a certain level of the hierarchy, such as in sociological or genetic studies. Using the framework of normally distributed endpoints, these questions will be studied, using both analytic calculation as well as Monte Carlo simulation.

Computational Statistics & Data Analysis, 2008

Information in a statistical experiment is often measured through the determinant of its information matrix. Under first order normal linear models, the determinant of the information matrix of a two-level factorial experiment neither depends on where the experiment is centered, nor on how it is oriented, and balanced allocations are more informative than unbalanced ones with the same number of runs. In contrast, under binary response models, none of these properties hold. The performance of two-level experiments for binomial responses is explored by investigating the dependence of the determinant of their information matrix on their location, orientation, range, presence or absence of interactions and on the relative allocation of runs to support points, and in particular, on the type of fractionating involved. Conventional wisdom about two-level factorial experiments, which is deeply rooted on normal response models, does not apply to binomial models. In binary response settings, factorial experiments should not be used for screening or as building blocks for binary response surface exploration, and there is no alternative to the optimal design theory approach to planning experiments. *

Journal of Statistical Computation and Simulation, 2005

By means of a fractional factorial simulation experiment, we compare the performance of Penalised Quasi-Likelihood, Non-Adaptive Gaussian Quadrature and Adaptive Gaussian Quadrature in estimating parameters for multi-level logistic regression models. The comparison is done in terms of bias, mean squared error, numerical convergence, and computational efficiency. It turns out that, in terms of Mean Squared Error, standard versions of the Quadrature methods perform relatively poor in comparison with Penalized Quasi-Likelihood.