Skip to main content

Duncan Watts

Followers

5

Following

1

Co-author

1

Public Views

Uploads

Papers by Duncan Watts

Scaling up experimental social, behavioral, and economic science

The standard experimental paradigm in the social, behavioral, and economic sciences is extremely ... more The standard experimental paradigm in the social, behavioral, and economic sciences is extremely limited. Although recent advances in digital technologies and crowdsourcing services allow individual experiments to be deployed and run faster than in traditional physical labs, a majority of experiments still focus on one-off results that do not generalize easily to real-world contexts or even to other variations of the same experiment. As a result, there exist few universally acknowledged findings, and even those are occasionally overturned by new data. We argue that to achieve replicable, generalizable, scalable and ultimately useful social and behavioral science, a fundamental rethinking of the model of virtual-laboratory style experiments is required. Not only is it possible to design and run experiments that are radically different in scale and scope than was possible in an era of physical labs; this ability allows us to ask fundamentally different types of questions than have bee...

Evaluating the scale, growth, and origins of right-wing echo chambers on YouTube

ArXiv, 2020

Although it is understudied relative to other social media platforms, YouTube is arguably the lar... more Although it is understudied relative to other social media platforms, YouTube is arguably the largest and most engaging online media consumption platform in the world. Recently, YouTube's outsize influence has sparked concerns that its recommendation algorithm systematically directs users to radical right-wing content. Here we investigate these concerns with large scale longitudinal data of individuals' browsing behavior spanning January 2016 through December 2019. Consistent with previous work, we find that political news content accounts for a relatively small fraction (11%) of consumption on YouTube, and is dominated by mainstream and largely centrist sources. However, we also find evidence for a small but growing "echo chamber" of far-right content consumption. Users in this community show higher engagement and greater "stickiness" than users who consume any other category of content. Moreover, YouTube accounts for an increasing fraction of these user...

Measuring the news and its impact on democracy

Proceedings of the National Academy of Sciences, 2021

Since the 2016 US presidential election, the deliberate spread of misinformation online, and on s... more Since the 2016 US presidential election, the deliberate spread of misinformation online, and on social media in particular, has generated extraordinary concern, in large part because of its potential effects on public opinion, political polarization, and ultimately democratic decision making. Recently, however, a handful of papers have argued that both the prevalence and consumption of “fake news” per se is extremely low compared with other types of news and news-relevant content. Although neither prevalence nor consumption is a direct measure of influence, this work suggests that proper understanding of misinformation and its effects requires a much broader view of the problem, encompassing biased and misleading—but not necessarily factually incorrect—information that is routinely produced or amplified by mainstream news organizations. In this paper, we propose an ambitious collective research agenda to measure the origins, nature, and prevalence of misinformation, broadly construe...

Computational Social Science

Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016

The past 15 years have witnessed a remarkable increase in both the scale and scope of social and ... more The past 15 years have witnessed a remarkable increase in both the scale and scope of social and behavioral data available to researchers, leading some to herald the emergence of a new field: "computational social science." Against these exciting developments stands a stubborn fact: that in spite of many thousands of published papers, there has been surprisingly little progress on the "big" questions that motivated the field in the first place?questions concerning systemic risk in financial systems, problem solving in complex organizations, and the dynamics of epidemics or social movements, among others. In this talk I highlight some examples of research that would not have been possible just a handful of years ago and that illustrate the promise of CSS. At the same time, they illustrate its limitations. I then conclude with some thoughts on how CSS can bridge the gap between its current state and its potential.

Comparing Estimates of News Consumption from Survey and Passively Collected Behavioral Data

Public Opinion Quarterly, 2021

Surveys are a vital tool for understanding public opinion and knowledge, but they can also yield ... more Surveys are a vital tool for understanding public opinion and knowledge, but they can also yield biased estimates of behavior. Here we explore a popular and important behavior that is frequently measured in public opinion surveys: news consumption. Previous studies have shown that television news consumption is consistently overreported in surveys relative to passively collected behavioral data. We validate these earlier findings, showing that they continue to hold despite large shifts in news consumption habits over time, while also adding some new nuance regarding question wording. We extend these findings to survey reports of online and social media news consumption, with respect to both levels and trends. Third, we demonstrate the usefulness of passively collected data for measuring a quantity such as “consuming news” for which different researchers might reasonably choose different definitions. Finally, recognizing that passively collected data suffers from its own limitations,...

Empirica: a virtual lab for high-throughput macro-level experiments

Behavior Research Methods, 2021

Virtual labs allow researchers to design high-throughput and macro-level experiments that are not... more Virtual labs allow researchers to design high-throughput and macro-level experiments that are not feasible in traditional in-person physical lab settings. Despite the increasing popularity of online research, researchers still face many technical and logistical barriers when designing and deploying virtual lab experiments. While several platforms exist to facilitate the development of virtual lab experiments, they typically present researchers with a stark trade-off between usability and functionality. We introduce Empirica: a modular virtual lab that offers a solution to the usability–functionality trade-off by employing a “flexible defaults” design strategy. This strategy enables us to maintain complete “build anything” flexibility while offering a development platform that is accessible to novice programmers. Empirica’s architecture is designed to allow for parameterizable experimental designs, reusable protocols, and rapid development. These features will increase the accessibil...

Integrating explanation and prediction in computational social science

Nature, 2021

A Large-Scale Comparative Study of Informal Social Networks in Firms

Management Science, 2021

Theories of organizations are sympathetic to long-standing ideas from network science that organi... more Theories of organizations are sympathetic to long-standing ideas from network science that organizational networks should be regarded as multiscale and capable of displaying emergent properties. However, the historical difficulty of collecting individual-level network data for many (N ≫ 1) organizations, each of which comprises many (n ≫ 1) individuals, has hobbled efforts to develop specific, theoretically motivated hypotheses connecting micro- (i.e., individual-level) network structure with macro-organizational properties. In this paper we seek to stimulate such efforts with an exploratory analysis of a unique data set of aggregated, anonymized email data from an enterprise email system that includes 1.8 billion messages sent by 1.4 million users from 65 publicly traded U.S. firms spanning a wide range of sizes and 7 industrial sectors. We uncover wide heterogeneity among firms with respect to all measured network characteristics, and we find robust network and organizational vari...

Collective Problem-Solving of Groups Across Tasks of Varying Complexity

As organizations gravitate to group-based structures, the problem of improving performance throug... more As organizations gravitate to group-based structures, the problem of improving performance through judicious selection of group members has preoccupied scientists and managers alike. However, it remains poorly understood under what conditions groups outperform comparable individuals, which individual attributes best predict group performance, or how task complexity mediates these relationships. Here we describe a novel two-phase experiment in which individuals were evaluated on a series of tasks of varying complexity; then randomly assigned to solve similar tasks either in groups of different compositions or as individuals. We describe two main sets of findings. First, while groups are more efficient than individuals and comparable “nominal group” when the task is complex, this relationship is reversed when the task is simple. Second, we find that average skill level dominates all other factors combined, including social perceptiveness, skill diversity, and diversity of cognitive st...

Measuring the predictability of life outcomes with a scientific mass collaboration

Proceedings of the National Academy of Sciences, 2020

How predictable are life trajectories? We investigated this question with a scientific mass colla... more How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life outcomes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.

Predicting history

Nature Human Behaviour, 2019

Can events be accurately described as historic at the time they are happening? Claims of this sor... more Can events be accurately described as historic at the time they are happening? Claims of this sort are in effect predictions about the evaluations of future historians; that is, that they will regard the events in question as significant. Here we provide empirical evidence in support of earlier philosophical arguments 1 that such claims are likely to be spurious and that, conversely, many events that will one day be viewed as historic attract little attention at the time. We introduce a conceptual and methodological framework for applying machine learning prediction models to large corpora of digitized historical archives. We find that although such models can correctly identify some historically important documents, they tend to overpredict historical significance while also failing to identify many documents that will later be deemed important, where both types of error increase monotonically with the number of documents under consideration. On balance, we conclude that historical significance is extremely difficult to predict, consistent with other recent work on intrinsic limits to predictability in complex social systems 2,3. However, the results also indicate the feasibility of developing 'artificial archivists' to identify potentially historic documents in very large digital corpora. Almost two centuries ago, Hegel 4 observed that "the owl of Minerva spreads its wings only with the falling of the dusk", meaning that philosophical understanding of an era is possible only as it is ending. Arthur Danto, an analytical philosopher, developed this intuition into a theoretical argument for why the judgement of history is intrinsically unpredictable 1. Danto imagined a hypothetical entity-the ideal chronicler-that possesses complete information about the state of the world, along with its entire history up to that point, and unlimited ability to integrate and analyse that information. Danto then argued that the ideal chronicler would still be unable to anticipate the significance that will be assigned to contemporaneous events by future historians because their judgements are inevitably informed by events that have not yet taken place. Evaluating the historical significance of a single event would in principle require the ideal chronicler to accurately predict not only its impact on subsequent events, but also how those events would impact broader social structures and practices 5. For example, to have anticipated the historical significance of the storming of the Bastille on 14 July 1789, the ideal chronicler would have had to predict not only the whole sequence of events leading to the French Revolution, but also their transformation of political and philosophical notions of sovereignty-a profound structural shift that has arguably shaped all subsequent revolutionary movements 6. Danto's argument is compelling in theory, but as a practical matter it may still be the case that some, or even most, historically significant events could be identified at the time they are happening or shortly thereafter. For example, while the full significance of the Bastille took many years to develop, contemporaneous accounts

Explanation, prediction, and causality: Three sides of the same coin?

In this essay we make four interrelated points. First, we reiterate previous arguments (Kleinberg... more In this essay we make four interrelated points. First, we reiterate previous arguments (Kleinberg et al 2015) that forecasting problems are more common in social science than is often appreciated. From this observation it follows that social scientists should care about predictive accuracy in addition to unbiased or consistent estimation of causal relationships. Second, we argue that social scientists should be interested in prediction even if they have no interest in forecasting per se. Whether they do so explicitly or not, that is, causal claims necessarily make predictions; thus it is both fair and arguably useful to hold them accountable for the accuracy of the predictions they make. Third, we argue that prediction, used in either of the above two senses, is a useful metric for quantifying progress. Important differences between social science explanations and machine learning algorithms notwithstanding, social scientists can still learn from approaches like the Common Task Fram...

Should social science be more solution-oriented?

Nature Human Behaviour, 2017

Prediction and explanation in social systems

Science (New York, N.Y.), Jan 3, 2017

Historically, social scientists have sought out explanations of human and social phenomena that p... more Historically, social scientists have sought out explanations of human and social phenomena that provide interpretable causal mechanisms, while often ignoring their predictive accuracy. We argue that the increasingly computational nature of social science is beginning to reverse this traditional bias against prediction; however, it has also highlighted three important issues that require resolution. First, current practices for evaluating predictions must be better standardized. Second, theoretical limits to predictive accuracy in complex social systems must be better characterized, thereby setting expectations for what can be predicted or explained. Third, predictive accuracy and interpretability must be recognized as complements, not substitutes, when evaluating explanations. Resolving these three issues will lead to better, more replicable, and more useful social science.

An Experimental Study of Team Size and Performance on a Complex Task

PLOS ONE, 2016

The relationship between team size and productivity is a question of broad relevance across econo... more The relationship between team size and productivity is a question of broad relevance across economics, psychology, and management science. For complex tasks, however, where both the potential benefits and costs of coordinated work increase with the number of workers, neither theoretical arguments nor empirical evidence consistently favor larger vs. smaller teams. Experimental findings, meanwhile, have relied on small groups and highly stylized tasks, hence are hard to generalize to realistic settings. Here we narrow the gap between real-world task complexity and experimental control, reporting results from an online experiment in which 47 teams of size ranging from n = 1 to 32 collaborated on a realistic crisis mapping task. We find that individuals in teams exerted lower overall effort than independent workers, in part by allocating their effort to less demanding (and less productive) sub-tasks; however, we also find that individuals in teams collaborated more with increasing team size. Directly comparing these competing effects, we find that the largest teams outperformed an equivalent number of independent workers, suggesting that gains to collaboration dominated losses to effort. Importantly, these teams also performed comparably to a field deployment of crisis mappers, suggesting that experiments of the type described here can help solve practical problems as well as advancing the science of collective intelligence.

Estimating the Causal Impact of Recommendation Systems from Observational Data

Proceedings of the Sixteenth ACM Conference on Economics and Computation, 2015

Recommendation systems are an increasingly prominent part of the web, accounting for up to a thir... more Recommendation systems are an increasingly prominent part of the web, accounting for up to a third of all traffic on several of the world's most popular sites. Nevertheless, little is known about how much activity such systems actually cause over and above activity that would have occurred via other means (e.g., search) if recommendations were absent. Although the ideal way to estimate the causal impact of recommendations is via randomized experiments, such experiments are costly and may inconvenience users. In this paper, therefore, we present a method for estimating causal effects from purely observational data. Specifically, we show that causal identification through an instrumental variable is possible when a product experiences an instantaneous shock in direct traffic and the products recommended next to it do not. We then apply our method to browsing logs containing anonymized activity for 2.1 million users on Amazon.com over a 9 month period and analyze over 4,000 unique products that experience such shocks. We find that although recommendation click-throughs do account for a large fraction of traffic among these products, at least 75% of this activity would likely occur in the absence of recommendations. We conclude with a discussion about the assumptions under which the method is appropriate and caveats around extrapolating results to other products, sites, or settings.

The Structural Virality of Online Diffusion

Management Science, 2015

Viral products and ideas are intuitively understood to grow through a person-to-person diffusion ... more Viral products and ideas are intuitively understood to grow through a person-to-person diffusion process analogous to the spread of an infectious disease; however, until recently it has been prohibitively difficult to directly observe purportedly viral events, and thus to rigorously quantify or characterize their structural properties. Here we propose a formal measure of what we label “structural virality” that interpolates between two conceptual extremes: content that gains its popularity through a single, large broadcast and that which grows through multiple generations with any one individual directly responsible for only a fraction of the total adoption. We use this notion of structural virality to analyze a unique data set of a billion diffusion events on Twitter, including the propagation of news stories, videos, images, and petitions. We find that across all domains and all sizes of events, online diffusion is characterized by surprising structural diversity; that is, popular...

Dissecting the Spirit of Gezi: Influence vs. Selection in the Occupy Gezi Movement

SSRN Electronic Journal, 2015

Do social movements actively shape the opinions and attitudes of participants by bringing togethe... more Do social movements actively shape the opinions and attitudes of participants by bringing together diverse groups that subsequently influence one another? Ethnographic studies of the 2013 Gezi uprising seem to answer "yes, " pointing to solidarity among groups that were traditionally indifferent, or even hostile, to one another. We argue that two mechanisms with differing implications may generate this observed outcome: "influence" (change in attitude caused by interacting with other participants); and "selection" (individuals who participated in the movement were generally more supportive of other groups beforehand). We tease out the relative importance of these mechanisms by constructing a panel of over 30,000 Twitter users and analyzing their support for the main Turkish opposition parties before, during, and after the movement. We find that although individuals changed in significant ways, becoming in general more supportive of the other opposition parties, those who participated in the movement were also significantly more supportive of the other parties all along. These findings suggest that both mechanisms were important, but that selection dominated. In addition to our substantive findings, our paper also makes a methodological contribution that we believe could be useful to studies of social movements and mass opinion change more generally. In contrast with traditional panel studies, which must be designed and implemented prior to the event of interest, our method relies on ex post panel construction, and hence can be used to study unanticipated or otherwise inaccessible events. We conclude that despite the well known limitations of social media, their "always on" nature and their widespread availability offer an important source of public opinion data.

Using the web to do social science

Proceedings of the 18th ACM international conference on Multimedia, 2010

What makes Social Science "Social"? • Social phenomena arise when individuals interact to produce... more What makes Social Science "Social"? • Social phenomena arise when individuals interact to produce collective entities that have their own attributes, rules etc.-Families, firms, markets, associations, societies, cultures • Systemic risk in financial systems, changes in cultural norms, collapse of political regimes Networked Public Goods S.Suri and D. J. Watts "A Study of Cooperation and Contagion in Networked Public Goods Experiments" (submitted) W. Mason and S. Suri. "A Guide to Running Experiments on AMT" (submitted)

Empirical agent based models of cooperation in public goods games

Proceedings of the fourteenth ACM conference on Electronic commerce, 2013

Agent-based models are a popular way to explore the dynamics of human interactions, but rarely ar... more Agent-based models are a popular way to explore the dynamics of human interactions, but rarely are these models based on empirical observations of actual human behavior. Here we exploit data collected in an experimental setting where over 150 human players played in a series of almost a hundred public goods games. First, we fit a series of deterministic models to the data, finding that a reasonably parsimonious model with just three parameters performs extremely well on the standard test of predicting average contributions. This same model, however, performs extremely poorly when predicting the full distribution of contributions, which is strongly bimodal. In response, we introduce and test a corresponding series of stochastic models, thus identifying a model that both predicts average contribution and also the full distribution. Finally, we deploy this model to explore hypotheses about regions of the parameter space outside of what was experimentally accessible. In particular, we investigate (a) whether a previous conclusion that network topology does not impact contribution levels holds for much larger networks than could be studied in a lab; (b) to what extent observed contributions depend on average network degree and variance in the degree distribution, and (c) the dependency of contributions on degree assortativity as well as the correlation between the generosity of players and the degree of the nodes to which they are assigned.