Academia.eduAcademia.edu

Outline

Title
Abstract
Figures
Choosing Cases for Case Studies
Case Studies Selection and Research Agendas
Using Case Studies to Evaluate Scope Conditions
Conclusion
References

Symposium: Case Selection, Case Studies and Causal Inference

Profile image of David CollierDavid Collier

Abstract

For scholars concerned with causal inference, how should cases be selected? The statistician David Freedman evaluates case selection strategies proposed by political scientists. He dissents from Fearon and Laitin, who suggest that when case studies are used to deepen findings derived from regression analysis, cases should be selected at random. Freedman also urges caution regarding Gerring’s approach, as he claims Gerring inappropriately proposes to identify cases as typical, diverse, extreme, etc. on the basis of criteria associated with large-N and experimental methods. Freedman likewise disagrees with Goertz’s recommendation about case-selection within the 2 x 2 matrix formed by dichotomous independent and dependent variables. Goertz suggests that analysts should ignore the 'null cell,' in which both the purported cause and the effect are absent, but Freedman argues against this practice, contending that all cells in the matrix can potentially be important. Through this discussion, these authors come to agree on the importance of grasping the strengths and limitations of alternative case-selection strategies.

Figures (1)
Figure 1: Descriptive-Causal Patterns: Wealth and Democracy
Figure 1: Descriptive-Causal Patterns: Wealth and Democracy

Related papers

The Role of Case Study Research in Political Science: Evidence for Causal Claims
Sharon Crasnow

Philosophy of Science, 2012

Political science research, particularly in international relations and comparative politics,has increasingly become dominated by statistical and formal approaches. The promise of these approaches shifted the methodological emphasis away from case study research. In response, supporters of case study research argue that case studies provide evidence for causal claims that is not available through statistical and formal research methods, and many have advocated multimethod research. I propose a way of understanding the integration of multiple methodologies in which the causes sought in case studies are treated as singular causation and contingent on a theoretical framework.

View PDFchevron_right
The case survey method and applications in political science. APSA 2009 Paper. Available at SSRN: http://ssrn.com/abstract=1451643
Jens Newig

2009

This paper reviews existing applications of case-based meta-analytic methods in political science and related fields and critically assesses strengths and limitations of the approach for political science. To this end, the paper proceeds as follows: The following chapter (2) places case study meta-analysis in the broader context of methods of secondary analysis that synthesize previous research. We develop a typology of methods according to the respective source of data as well as the method of integration. Subsequently, review previous applications of the case survey method in political research and related fields and discuss the relevance of these findings as well as methodological innovations as well as flaws . We then portray the case survey method (case-based meta-analysis) in more detail, drawing on experiences from our own research (4). From these, we draw conclusions for the applicability of case meta-analysis, discussing strengths and weaknesses of the method specifically related to political research and conclude with an outlook for further methodological improvements in the field of meta-analytical and other evidencebased methods (5).

View PDFchevron_right
Causal inference and American political development: contrasts and complementarities
Jeffery Jenkins

Public Choice, 2019

The causal inference (CI) movement has forced political scientists to think far more seriously about what can be learned from a particular research design and to be more attentive to making design choices that allow for credible causal inferences. At the same time, the rise of CI has given rise to the concern that political scientists have been better at making particular contributions isolating the effect of a single independent variable than at developing and testing theories that help us understand how these diverse findings fit together. For all of American political development's (APD's) distance from the causal inference revolution, a parallel can be drawn between the state of APD today and some of the concerns expressed about the broader state of political science in the wake of the rise of CI. This essay considers ways in which APD and CI can each be enriched through greater mutual engagement, suggesting that one should not dichotomize research into mutually exclusive categories of "well-identified research that makes credible causal claims" and "purely descriptive" APD studies.

View PDFchevron_right
Qualitative Causal inference and Critical Junctures: The Problem of Backdoor Paths
David Waldner

Critical Junctures and Historical Legacies: Insights and Methods for Comparative Social Sciences, 2022

View PDFchevron_right
Case study research and causal inference
Benjamin Hanckel

BMC Medical Research Methodology, 2022

Case study methodology is widely used in health research, but has had a marginal role in evaluative studies, given it is often assumed that case studies offer little for making causal inferences. We undertook a narrative review of examples of case study research from public health and health services evaluations, with a focus on interventions addressing health inequalities. We identified five types of contribution these case studies made to evidence for causal relationships. These contributions relate to: (1) evidence about system actors' own theories of causality; (2) demonstrative examples of causal relationships; (3) evidence about causal mechanisms; (4) evidence about the conditions under which causal mechanisms operate; and (5) inference about causality in complex systems. Case studies can and do contribute to understanding causal relationships. More transparency in the reporting of case studies would enhance their discoverability, and aid the development of a robust and pluralistic evidence base for public health and health services interventions. To strengthen the contribution that case studies make to that evidence base, researchers could: draw on wider methods from the political and social sciences, in particular on methods for robust analysis; carefully consider what population their case is a case 'of'; and explicate the rationale used for making causal inferences.

View PDFchevron_right
MECHANISMS, CAUSAL MODELING, AND THE LIMITATIONS OF TRADITIONAL MULTIPLE REGRESSION
harold kincaid
View PDFchevron_right
Statistical variable selection and causality in the social and behavioral sciences
harold kincaid

Quality and Quantity, 2024

The problem of "variable selection" is a fundamental one across the sciences. In its broadest terms, this problem would be at least part of the general issue of theory selection and comparison. However, there is a more circumscribed problem that concerns primarily the choice of variables for the best fitting model, given some set of data, usually observational in nature, and specific statistical techniques, typically multiple regression. There is a deep strand in econometrics and other applied social, behavioral, and biomedical science statistics to want formal decision rules or algorithms to pick out variables. The paper examines seven such formal procedures using a simulated data set with known causal relations. The conclusion is that seven often-used procedures make systematic causal errors. Some suggestions about better alternatives conclude.

View PDFchevron_right
On making causal claims: A review and recommendations
Rafael Lalive

Leadership Quarterly, 2010

a r t i c l e i n f o a b s t r a c t Social scientists often estimate models from correlational data, where the independent variable has not been exogenously manipulated; they also make implicit or explicit causal claims based on these models. When can these claims be made? We answer this question by first discussing design and estimation conditions under which model estimates can be interpreted, using the randomized experiment as the gold standard. We show how endogeneitywhich includes omitted variables, omitted selection, simultaneity, common-method variance, and measurement errorrenders estimates causally uninterpretable. Second, we present methods that allow researchers to test causal claims in situations where randomization is not possible or when causal interpretation could be confounded; these methods include fixed-effects panel, sample selection, instrumental variable, regression discontinuity, and difference-in-differences models. Third, we take stock of the methodological rigor with which causal claims are being made in a social sciences discipline by reviewing a representative sample of 110 articles on leadership published in the previous 10 years in top-tier journals. Our key finding is that researchers fail to address at least 66% and up to 90% of design and estimation conditions that make causal claims invalid. We conclude by offering 10 suggestions on how to improve non-experimental research.

View PDFchevron_right
Co-Variation and Causal Process Tracing Revisited: Clarifying New Directions for Causal Inference and Generalization in Case Study Methodology
Joachim Blatter

2008

ion Drawing conclusions from adequacy for the 1. Specification and justification of the range of empirical case(s) to the relavance/relative theories which are applied strength of theories within the broader 2. Selecting cases according to their “likeliness” scientific discourse for the dominant theory son within the case study proper ceases to play a substantial role within the co-variational template. This is due to the fact that the authors dismiss the most-different cases technique and that all other techniques are geared to select only one case. In consequence, Gerring and Seawright in fact challenge an assumption that has become (implicitly or explicitly) commonsensical, especially in comparative politics: the assumption that analyzing and comparing a few cases in-depth is better than analyzing one case. This leaves us with the following puzzle: If we want to generalize towards a population and if–given this goal–the cross-case comparison must include (a representative sample ...

View PDFchevron_right
Mostly Useless Econometrics? Assessing the Causal Effect of Econometric Theory
John Rust

Foundations and Trends® in Accounting

When it comes to causal conclusions, rigor matters. To this end we impose high standards for how studies from which we draw causal conclusions are conducted. For instance, we are widely urged to prefer randomized controlled trials (RCTs) or instrumental variable (IV) models to observational studies relying just on correlations, and we have explicit criteria for what counts as a good RCT or a good IV model. But we tend to be shockingly sloppy when it comes to making explicit just what the causal conclusions we draw mean, why the methods we employ are good for establishing conclusions with just that meaning, and what can defensibly be taken to follow from these claims. With respect to what can be inferred from the limited causal conclusions our studies support, we are far too prone to over reach, to 'generalize' that what holds in a study or handful of studies holds widely. But, I shall argue, we do not get arrant for general claims by generalizing. Rather it takes a great tangle of scientific work to support a general claim, including a great deal of conceptual development, theory and the confirmation of a variety of different kinds of effects that the general claim implies.

View PDFchevron_right

References (62)

  1. Box-Steffensmeier, Janet M., Henry E. Brady, and David Collier, eds. 2008. Oxford Handbook of Political Methodology. Oxford: Oxford University Press.
  2. Campbell, Donald T. 1975. "'Degrees of Freedom' and the Case Study." Comparative Political Studies 8:2 (July), 178-93.
  3. Eckstein, Harry. 1975. "Case Study and Theory in Political Sci- ence." In Fred I. Greenstein and Nelson W. Polsby, eds., Hand- book of Political Science, Vol. 7. Reading, MA: Addison-Wesley.
  4. Fearon, James D. and David D. Laitin. 2008. "Integrating Qualitative and Quantitative Methods." In The Oxford Handbook of Political Methodology. Janet Box-Steffensmeier, Henry E. Brady, and David Collier, eds. (New York: Oxford University Press), 756-76.
  5. Freedman, David A. 2008. "On Types of Scientific Enquiry: The Role of Qualitative Reasoning." In The Oxford Handbook of Politi- cal Methodology. Janet Box-Steffensmeier, Henry E. Brady, and David Collier, eds. (New York: Oxford University Press), 300-18.
  6. George, Alexander L. 1979. "Case Studies and Theory Develop- ment: The Method of Structured, Focused Comparison." In Paul Gordon Lauren, ed., Diplomacy: New Approaches in History, Theory and Policy. New York: Free Press.
  7. Gerring, John. 2008. "Case Selection for Case-Study Analysis: Quali- tative and Quantitative Techniques." In The Oxford Handbook of Political Methodology. Janet Box-Steffensmeier, Henry E. Brady, and David Collier, eds. (New York: Oxford University Press), 645-84.
  8. Lijphart, Arend. 1971. "Comparative Politics and the Comparative Method." American Political Science Review 65:3 (September): 682-93.
  9. Mill, John Stuart. 1974b [1843]. "Of the Four Methods of Experi mental Inquiry." In Book 3, Chapter 8, A System of Logic,Racio- cinative and Inductive. Toronto: University of Toronto Press.
  10. Przeworski, Adam, and Henry Teune. 1970. The Logic of Compara- tive Social Inquiry. New York: John Wiley.
  11. Ragin, Charles C. 1987. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley: University of 691-713.
  12. Freedman, David A. 2008a. "On Types of Scientific Enquiry: The Role of Qualitative Reasoning." In The Oxford Handbook of Politi- cal Methodology. Janet Box-Steffensmeier, Henry E. Brady, and David Collier, eds. (New York: Oxford University Press), 300-18.
  13. Freedman, David A. 2008b. "On Regression Adjustments in Experi- ments with Several Treatments." Annals of Applied Statistics 2: 176-96.
  14. Freedman, David A. 2008c. "Randomization Does Not Justify Logis- tic Regression." Statistical Science 23: 237-49.
  15. Freedman, David A. 2008d. "Survival Analysis: A Primer." The Ameri- can Statistician 62: 110-19.
  16. Freedman, David A. 2008e. "Oasis or Mirage?" Chance 21: 59-61.
  17. Freedman, David A., Robert Pisani, and Roger A. Purves. 2007. Sta- tistics. 4th edition. New York: W. W. Norton & Company, Inc.
  18. Gerring, John . 2008. "Case Selection for Case-Study Analysis: Quali- tative and Quantitative Techniques." In The Oxford Handbook of Political Methodology. Janet Box-Steffensmeier, Henry E. Brady, and David Collier, eds. (New York: Oxford University Press), 645- 84.
  19. Glazerman, Steven, Dan M. Levy, and David Myers. 2003. "Non- experimental versus Experimental Estimates of Earnings Impacts." Annals of the American Academy of Political and Social Science 589: 63-93.
  20. King, Gary, Robert O. Keohane, and Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Prince- ton: Princeton University Press.
  21. Mahoney, James and Dietrich Rueschemeyer. 2003. Comparative Historical Analysis in the Social Sciences. New York: Cambridge University Press.
  22. Peikes, Deborah N., Lorenzo Moreno, and Sean Michael Orzol. 2008. "Propensity Score Matching: A Note of Caution for Evaluators of Social Programs." The American Statistician 62: 222-31.
  23. Seawright, Jason and John Gerring. 2008. "Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options." Political Research Quarterly 61: 294-308.
  24. Wilde, Elizabeth T. and Robinson Hollister. 2007. "How Close Is Close Enough? Evaluating Propensity Score Matching Using Data from a Class Size Reduction Experiment." Journal of Policy Analy- sis and Management 26: 455-77.
  25. Arceneaux, Kevin, Alan S. Gerber, and Donald P. Green. 2006. "Com- paring Experimental and Matching Methods Using a Large-Scale Voter Mobilization Experiment." Political Analysis 14: 37-62.
  26. Berk, Richard A. 2004. Regression Analysis: A Constructive Critique. Sage Publications.
  27. Box-Steffensmeier, Janet M., Henry E. Brady, and David Collier. 2008. The Oxford Handbook of Political Methodology. New York: Oxford University Press.
  28. Brady, Henry E. and David Collier. 2004. Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham, MD: Rowman and Littlefield.
  29. Fearon, James D. and David D. Laitin. 2008. "Integrating Qualitative and Quantitative Methods." In The Oxford Handbook of Political Methodology. Janet Box-Steffensmeier, Henry E. Brady, and David Collier, eds. (New York: Oxford University Press), 756-76.
  30. Freedman, David A. 2005. Statistical Models: Theory and Practice. New York: Cambridge University Press.
  31. Freedman, David A. 2006. "Statistical Models for Causation: What Inferential Leverage Do They Provide?" Evaluation Review 30: References
  32. Eckstein, Harry. 1975. "Case Studies and Theory in Political Sci- ence." In Handbook of Political Science, vol. 7. Political Science: Scope and Theory. Fred I. Greenstein and Nelson W. Polsby, eds. (Reading, MA: Addison-Wesley), 94-137.
  33. Gerring, John. 2007a. Case Study Research: Principles and Practices. Cambridge: Cambridge University Press.
  34. Gerring, John. 2007b. "The Case Study: What it is and What it Does." In Oxford Handbook of Comparative Politics. Carles Boix and Susan Stokes, eds. (New York: Oxford University Press), 90- 122.
  35. Gerring, John and Rose McDermott. 2007. "An Experimental Tem- plate for Case-Study Research." American Journal of Political Science 51:3 (July), 688-701.
  36. Seawright, Jason and John Gerring. 2008. "Case-Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options." Political Research Quarterly 61:2 (June), 294-308.
  37. Diamond, Larry. 1992. "Economic Development and Democracy Re- considered." In Gary Marks and Larry Diamond, eds. Reexamin- ing Democracy: Essays in Honor of Seymour Martin Lipset. (New- bury Park, CA: Sage Publications).
  38. Dul, Jan and Tony Hak. 2008. Case Study Methodology in Business Research. Amsterdam: Elsevier.
  39. Fearon, James and David Laitin. 2008. "Integrating Qualitative and Quantitative Methods." In Janet Box-Steffensmier, Henry Brady, and David Collier, eds. The Oxford Handbook of Political Method- ology. Oxford: Oxford University Press.
  40. George, Alexander and Andrew Bennett. 2005. Case Studies and Theory Development. Cambridge: MIT Press.
  41. Gerring, John. 2007. Case Study Research: Principles and Practices. Cambridge: Cambridge University Press.
  42. Gerring, John. 2008. "Case Selection for Case Study Analysis: Quali- tative and Quantitative Techniques." In Janet Box-Steffensmier, Henry Brady, and David Collier, eds. The Oxford Handbook of Political Methodology. Oxford: Oxford University Press.
  43. King, Gary, Robert Keohane, and Sidney Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton: Princeton University Press.
  44. King, Gary and Langche Zeng. 2007. "When Can History be our Guide? The Pitfalls of Counterfactual Inference." International Studies Quarterly 51:183-210.
  45. Levy, Jack S. 2008. "Counterfactuals and Case Studies." In Janet Box-Steffensmier, Henry Brady, and David Collier, eds. The Ox- ford Handbook of Political Methodology. Oxford: Oxford Uni- versity Press.
  46. Lieberman, Evan. 2005. "Nested Analysis as a Mixed-Method Strat- egy for Comparative Research." American Political Science Re- view 99: 435-52.
  47. Lipset, Seymour Martin. 1994. "Social Requisites of Democracy Revisited." American Sociological Review 59: 1-22.
  48. Mahoney, James and Gary Goertz. 2004. "The Possibility Principle: Choosing Negative Cases in Comparative Research." American Political Science Review 98: 653-69.
  49. Mahoney, James and Gary Goertz. 2006. "A Tale of Two Cultures: Contrasting Quantitative and Qualitative Research." Political Analy- sis 14: 227-49.
  50. Przeworski, Adam, et al. 2000. Democracy and Development: Politi- cal Institutions and Well-being in the World, 1950-1990. Cam- bridge: Cambridge University Press.
  51. Ragin, Charles. 1987. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley: University of California Press.
  52. Ragin, Charles. 2000. Fuzzy-Set Social Science. Chicago: University of Chicago Press.
  53. Ragin, Charles. 2008. Redesigning Social Inquiry: Fuzzy Sets and Beyond. Chicago: University of Chicago Press.
  54. Tetlock, Philip E. and Aaron Belkin, eds. 1996. Counterfactual Thought Experiments in World Politics: Logical, Methodological, and Psychological Perspectives. Princeton: Princeton University Press.
  55. Ulfelder, Jay. 2007. "Natural Resource Wealth and the Survival of Autocracy." Comparative Political Studies 40: 995-1018. References
  56. Box-Steffensmeier, Janet M., Henry E. Brady, and David Collier. 2008. The Oxford Handbook of Political Methodology. Oxford Uni- versity Press.
  57. Fearon, James and David Laitin. 2008. "Integrating Qualitative and Quantitative Methods." In Janet Box-Steffensmier, Henry Brady, and David Collier, eds. The Oxford Handbook of Political Method- ology (Oxford: Oxford University Press), 756-76.
  58. Freedman, David A. 1999. "From Association to Causation: Some Remarks on the History of Statistics." Statistical Science 14: 243- 58. Reprinted in Journal de la Société Française de Statistique 40 (1999), 5-32 and in John Panaretos, ed. 2003. Stochastic Musings: Pespectives from the Pioneers of the Late 20th Century. Lawrence Erlbaum Associates, 45-71.
  59. Freedman, David A. 2005. Statistical Models: Theory and Practice. Cambridge University Press.
  60. Freedman, David A., Robert Pisani, and Roger A. Purves. 2007. Sta- tistics. 4th edition. New York: W. W. Norton & Company, Inc.
  61. Gerring, John. 2008. "Case Selection for Case Study Analysis: Quali- tative and Quantitative Techniques." In Janet Box-Steffensmier, Henry Brady, and David Collier, eds. The Oxford Handbook of Political Methodology (Oxford: Oxford University Press), 645- 84. Gordis, Leon. 2008. Epidemiology. 4th edition. Philadelphia: Elsevier- Saunders.
  62. Seawright, Jason and John Gerring. 2008. "Case Selection Techniques in Case Study Research: A Menu of Qualitative and Quantitative Options." Political Research Quarterly 61: 294-308.

Related papers

The Statistics of Causal Inference: The View from Political Methodology
L. Keele

Many areas of political science focus on causal questions. Evidence from statistical analyses are often used to make the case for causal relationships. While statistical analyses can help establish causal relationships, it can also provide strong evidence of causality where none exists. In this essay, I provide an overview of the statistics of causal inference. Instead of focusing on specific statistical methods, such as matching, I focus more on the assumptions needed to give statistical estimates a causal interpretation. Such assumptions are often referred to as identification assumptions, and these assumptions are critical to any statistical analysis about causal effects. I outline a wide range of identification assumptions and highlight the design-based approach to causal inference. I conclude with an overview of statistical methods that are frequently used for causal inference. . For comments I thank the editors and the four anonymous reviewers. I also thank Rocío Titiunik, Jasjeet Sekhon, Paul Rosenbaum, and Dylan Small for many insightful conversations about these topics over the years. In the online Supplementary Materials, I provide further information about software tools to implement many of the methodologies discussed in this essay. sion Discontinuity Estimator." Review of Economic Studies 79 :933-959. Kang, Joseph DY, Joseph L Schafer et al. 2007. "Demystifying double robustness: A comparison of alternative strategies for estimating a population mean from incomplete data." Statistical science 22(4):523-539.

View PDFchevron_right
Taking Causal Heterogeneity Seriously: Implications for Case Choice and Case Study-Based Generalizations
Steffen Hertog

Sociological Methods & Research

In mixed methods approaches, statistical models are used to identify “nested” cases for intensive, small- n investigation for a range of purposes, including notably the examination of causal mechanisms. This article shows that under a commonsense interpretation of causal effects, large- n models allow no reliable conclusions about effect sizes in individual cases—even if we choose “onlier” cases as is usually suggested. Contrary to established practice, we show that choosing “reinforcing” outlier cases—where outcomes are stronger than predicted in the statistical model—is appropriate for testing preexisting hypotheses on causal mechanisms, as this reduces the risk of false negatives. When investigating mechanisms inductively, researchers face a choice between “onlier” and reinforcing outlier cases that represents a trade-off between false negatives and false positives. We demonstrate that the inferential power of nested research designs can be much increased through paired compariso...

View PDFchevron_right
The Statistics of Causal Inference in the Social Sciences Political Science C236A Statistics C239A
Danny Hidalgo

Approaches to causal inference using the potential outcomes framework. Covers observational studies with and without ignorable treatment assignment, randomized experiments with and without noncompliance, instrumental variables, regression discontinuity, sensitivity analysis and randomization inference. Applications are drawn from a variety of fields including political science, economics, sociology, public health and medicine.

View PDFchevron_right
Evidence for Use: Causal Pluralism and the Role of Case Studies in Political Science Research
Sharon Crasnow

Philosophy of the Social Sciences, 2010

Most contemporary political science researchers are advocates of multimethod research, however, the value and proper role of qualitative methodologies, like case study analysis, is disputed. A pluralistic philosophy of science can shed light on this debate. Methodological pluralism is indeed valuable, but does not entail causal pluralism. Pluralism about the goals of science is relevant to the debate and suggests a focus on the difference between evidence for warrant and evidence for use. I propose that case study research provides evidence for use through providing information that bears on the applicability of causal generalizations and risk assessment.

View PDFchevron_right
How the Cases You Choose Affect the Answers You Get: Selection Bias in Comparative Politics
Antonella Yosver

This article demonstrates how the selection of cases for study on the basis of outcomes on the dependent variable biases conclusions. It first lays out the logic of explanation and shows how it is violated when only cases that have achieved the outcome of interest are studied. It then examines three well-known and highly regarded studies in the field of comparative politics, comparing the conclusions reached in the original work with a test of the arguments on cases selected without regard for their position on the dependent variable. In each instance, conclusions based on the uncorrelated sample differ from the original conclusions. Comparative politics, like other subfields in political science, has norms and conventions about what constitutes an appropriate research strategy and what kind of evidence makes an argument persuasive. One of our most durable conventions is the selection of cases for study on the dependent variable. 1 That is, if we want to understand something, for example, revolution, we select one or more occurrences and subject them to scrutiny. Most graduate students learn in the statistics courses forced upon them that selection on the dependent variable is forbidden, but few remember why, or what the implications of violating this taboo are for their own work. And so, comparativists often ignore or forget about it when carrying out or assessing nonquantitative comparative research.

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved