In this paper we explore the differential perceptions of cybersecurity professionals and general ... more In this paper we explore the differential perceptions of cybersecurity professionals and general users regarding access rules and passwords. We conducted a preliminary survey involving 28 participants: 15 cybersecurity professionals and 13 general users. We present our preliminary findings and explain how such survey data might be used to improve security in practice. We focus on user fatigue with access rules and passwords. When asked "Who sets policy about access to the computers and systems (e.g., desktops, network, laptops, servers) you use
Agent-based modeling can serve as a valuable asset to security personnel who wish to better under... more Agent-based modeling can serve as a valuable asset to security personnel who wish to better understand the security landscape within their organization, especially as it relates to user behavior and circumvention. In this paper, we argue in favor of cognitive behavioral agent-based modeling for usable security and report on our work on developing an agent-based model for a password management scenario. We perform a number of trials and a sensitivity analysis that provide valuable insights into improving security (e.g., an organization that wishes to suppress one form of circumvention may want to endorse another form of circumvention). Agent-based models incorporating user behavior, emotion, and cognition can serve as valuable tools that assist computer security personnel design, implement, and maintain security systems, devise security policies, and employ security practices that are congruent with security and other organizational objectives. Indeed, as the current state of security practice indicates, we need these sorts of tools. Our interviews, surveys, and observations reflect many examples where security fails to accommodate users. Such mismatches between user needs and security policies and mechanisms often induce circumvention, thereby undermining overall objectives. Even if one could design adequate security policies and mechanisms a priori, the dynamic nature of software systems, user needs, and organizational and environmental changes would necessitate frequent readjustments. Consequently, we need tools that allow us to better understand computer security's costs, common perceptions and misperceptions, side effects, and interactions.
Keele and Small responded to our article on instrumental variables (IVs) published in Health Serv... more Keele and Small responded to our article on instrumental variables (IVs) published in Health Services Research in February 2017. Here, we address their efforts to defend IVs and we present additional evidence of the unreliability of IVs in comparative effectiveness research (CER). We appreciate that some economists, statisticians, and other IV adherents are emboldened by their faith in the power of weak cross-sectional associations to accurately reflect the world. But health outcomes research requires confronting the interrelatedness of social and medical factors-almost always a confounded reality with unmeasured and, indeed, unknown variables. 3 That is, most IV studies assume life is far less confounded than it is. Rassen et al 4 define an instrumental variable (IV) as "an unconfounded proxy for a study exposure that can be used to estimate a causal effect in the presence of unmeasured confounding." In other words, it assumes that a common variable (eg, differences between regions in rates of a medical treatment [eg, prostatectomy]) can assume randomization in estimating the effect on mortality-without having to worry about unknown confounding. In this wonderfully convenient but almost always unrealistic theory, IVs can randomize exposure to medical interventions. For example, IV proponents used distance from patients' homes to the hospital as an instrumental variable (IV) that presumably "randomizes" early MI treatments to estimate effects on mortality rates. In these examples, however, the IV is likely associated with other critical variables (eg, urban/ rural status, socioeconomic characteristics, health status) that impact health outcomes. The IV is confounded and therefore produces a biased effect estimate. Even after more than a quarter century of IV research in health care, most instrumental variables still violate the assumptions that the IV-Outcome relationship is not confounded. Studies that use IV also ignore or accept inadequate measures of countless confounders (eg, health status, clinical risk factors, geography, procedure volume, access to care, and SES-part of a longer list we discuss below).
Journal of General Internal Medicine, Oct 18, 2016
Some medical scientists argue that only data from randomized controlled trials (RCTs) are trustwo... more Some medical scientists argue that only data from randomized controlled trials (RCTs) are trustworthy. They claim data from natural experiments and administrative data sets are always spurious and cannot be used to evaluate health policies and other population-wide phenomena in the real world. While many acknowledge biases caused by poor study designs, in this article we argue that several valid designs using administrative data can produce strong findings, particularly the interrupted time series (ITS) design. Many policy studies neither permit nor require an RCT for cause-and-effect inference. Framing our arguments using Campbell and Stanley's classic research design monograph, we show that several Bquasi-experimental^designs, especially interrupted time series (ITS), can estimate valid effects (or non-effects) of health interventions and policies as diverse as public insurance coverage, speed limits, hospital safety programs, drug abuse regulation and withdrawal of drugs from the market. We further note the recent rapid uptake of ITS and argue for expanded training in quasiexperimental designs in medical and graduate schools and in post-doctoral curricula.
Infection Control and Hospital Epidemiology, May 1, 2007
Objective-Prior-approval antimicrobial stewardship programs (ASPs) improve patient outcomes and d... more Objective-Prior-approval antimicrobial stewardship programs (ASPs) improve patient outcomes and decrease antimicrobial resistance. These benefits would be limited if physicians circumvent ASP efforts. Our objectives were: 1) to determine if there is an increase in the proportion of orders for restricted (vs. non-restricted) antimicrobials in the first hour when priorapproval is not required compared with the remainder of the day, and 2) to determine if restricted antimicrobials ordered in the first hour when prior-approval is not required are less likely to be continued when the ASP resumes than antimicrobials ordered in the preceding hour. Design-A cross-sectional study design and a retrospective cohort study design, respectively. Setting-The study was set in a tertiary care academic medical center with a prior-approval ASP operational from 8 a.m.-10 p.m. Results-Compared with other hours, a greater proportion of orders during 10-10:59 p.m. were for restricted agents (57.0% vs. 49.9%, p=0.02). Surgical patients with orders placed between 10-10:59 p.m. (vs. 9-9:59 p.m.) were less likely to have the ordered antimicrobial continued (60.0% vs. 98.1%, p<0.001). Non-surgical patients with orders placed between 10-10:59 p.m. (vs. 9 -9:59 p.m.) were also less likely to have the ordered antimicrobial continued (70.8% vs. 84.2%, p=0.01). avoid prior-approval by waiting until restrictions are no longer in place to order restricted antimicrobials. These antimicrobials are less often continued when the ASP
We discuss our ongoing work with an agent-based password simulation which models how site-enforce... more We discuss our ongoing work with an agent-based password simulation which models how site-enforced password requirements affect aggregate security when people interact with multiple authentication systems. We model two password memorization techniques: passphrase generation and spaced repetition. Our simulation suggests system-generated passphrases lead to lower aggregate security across services that enforce even moderate password requirements. Furthermore, allowing users to expand their password length over time via spaced repetition increases aggregate security.
The usability of any security measure is often dependent on the environment and context in which ... more The usability of any security measure is often dependent on the environment and context in which it is deployed. A better understanding of context can help avoid a one-size-fits-all approach that can lead to security that is burdensome to use and does not address the most relevant vulnerabilities. A key aspect is a group's workflow -repeated group activities that selectively change the importance of vulnerabilities and that selectively restrict the time and cognitive budget available for security. Here we describe a number of case studies (drawn mainly from our own fieldwork) in which the workflow renders unusable a security approach that may be effective in other environments. We distinguish cases where the problem arises from individual tasks, from multiple paths through a workflow that may be unexpected, from time or cognitive stress introduced by the workflow and from barriers to passing needed information for the organization's mission. We present general approaches to design and improve upon security solutions so that they fit organizational workflow. Moreover, we discuss our ongoing efforts of conducting a broad cross-organization security-workflow oriented survey, cataloging and analyzing a wide-range of security failures and successes (many of which stem from workflow and security interactions), and agent-based simulation efforts.
Effective reasoning about the impact of security policy decisions requires understanding how huma... more Effective reasoning about the impact of security policy decisions requires understanding how human users actually behave, rather than assuming desirable but incorrect behavior. Simulation could help with this reasoning, but it requires building computational models of the relevant human behavior and validating that these models match what humans actually do. In this paper we describe our progress on building agent-based models of human behavior with passwords, and we demonstrate how these models reproduce phenomena shown in the empirical literature.
In this paper we explore the differential perceptions of cybersecurity professionals and general ... more In this paper we explore the differential perceptions of cybersecurity professionals and general users regarding access rules and passwords. We conducted a preliminary survey involving 28 participants: 15 cybersecurity professionals and 13 general users. We present our preliminary findings and explain how such survey data might be used to improve security in practice. We focus on user fatigue with access rules and passwords.
Observational studies of medical treatment effectiveness have increased substantially during the ... more Observational studies of medical treatment effectiveness have increased substantially during the last several decades (Garber 2011) in part due to the growing realization that randomized controlled trials, the presumed gold standard of such research, are often not generalizable to the real world (Koppel 2013). Moreover, the economic stimulus of 2009 spurred the application of new medical effectiveness research methods using observational data, such as instrumental variable (IV) analysis . In this issue of the journal, Sanwald and Schober (2016) analyzed the effects on survival of access to catheterization (cath) laboratories and invasive treatment of heart attack using an IV, distance to the hospital. We discuss the strengths and limitations of this study and demonstrate why this method more often produces untrustworthy estimates of the effects of medical treatments. IV analyses are statistical analyses, not research designs as articulated by their users. Many, weak research designs do not protect against bias even with heroic statistical adjustment to control for differences between the groups being studied . Unfortunately, most, but not all, IV studies use the weakest observational designs which do not demonstrate cause and effect. In the wise words of : "You can't fix by analysis what you bungled by design." WHAT IS AN INSTRUMENTAL VARIABLE? An instrumental variable (IV) is a variable, generally found in administrative data, that is assumed to randomize a treatment to estimate cause and effect
Proceedings of the 2015 Symposium and Bootcamp on the Science of Security, 2015
Agent-based modeling can serve as a valuable asset to security personnel who wish to better under... more Agent-based modeling can serve as a valuable asset to security personnel who wish to better understand the security landscape within their organization, especially as it relates to user behavior and circumvention. In this paper, we argue in favor of cognitive behavioral agent-based modeling for usable security and report on our work on developing an agent-based model for a password management scenario. We perform a number of trials and a sensitivity analysis that provide valuable insights into improving security (e.g., an organization that wishes to suppress one form of circumvention may want to endorse another form of circumvention). Agent-based models incorporating user behavior, emotion, and cognition can serve as valuable tools that assist computer security personnel design, implement, and maintain security systems, devise security policies, and employ security practices that are congruent with security and other organizational objectives. Indeed, as the current state of security practice indicates, we need these sorts of tools. Our interviews, surveys, and observations reflect many examples where security fails to accommodate users. Such mismatches between user needs and security policies and mechanisms often induce circumvention, thereby undermining overall objectives. Even if one could design adequate security policies and mechanisms a priori, the dynamic nature of software systems, user needs, and organizational and environmental changes would necessitate frequent readjustments. Consequently, we need tools that allow us to better understand computer security's costs, common perceptions and misperceptions, side effects, and interactions.
Proceedings of the 1st International Workshop on Agents and CyberSecurity, 2014
Security subsystems are often designed with flawed assumptions arising from system designers' fau... more Security subsystems are often designed with flawed assumptions arising from system designers' faulty mental models. Designers tend to assume that users behave according to some textbook ideal, and to consider each potential exposure/interface in isolation. However, fieldwork continually shows that even well-intentioned users often depart from this ideal and circumvent controls in order to perform daily work tasks, and that "incorrect" user behaviors can create unexpected links between otherwise "independent" interfaces. When it comes to security features and parameters, designers try to find the choices that optimize security utilityexcept these flawed assumptions give rise to an incorrect curve, and lead to choices that actually make security worse, in practice. We propose that improving this situation requires giving designers more accurate models of real user behavior and how it influences aggregate system security. Agentbased modeling can be a fruitful first step here. In this paper, we study a particular instance of this problem, propose user-centric techniques designed to strengthen the security of systems while simultaneously improving the usability of them, and propose further directions of inquiry.
Journal of the American Medical Informatics Association, 2020
The COVID-19 pandemic response in the United States has exposed significant gaps in information s... more The COVID-19 pandemic response in the United States has exposed significant gaps in information systems and processes that prevent timely clinical and public health decision-making. Specifically, the use of informatics to mitigate the spread of SARS-CoV-2, support COVID-19 care delivery, and accelerate knowledge discovery bring to the forefront issues of privacy, surveillance, limits of state powers, and interoperability between public health and clinical information systems. Using a consensus-building process, we critically analyze informatics-related ethical issues in light of the pandemic across 3 themes: (1) public health reporting and data sharing, (2) contact tracing and tracking, and (3) clinical scoring tools for critical care. We provide context and rationale for ethical considerations and recommendations that are actionable during the pandemic and conclude with recommendations calling for longer-term, broader change (beyond the pandemic) for public health organization and ...
The health information technology (HIT) implementation listserv was conceived as a way to combine... more The health information technology (HIT) implementation listserv was conceived as a way to combine a substantial portion of American Medical Informatics Association (AMIA) members who belonged to four working groups (WGs): CIS, Evaluation, ELSI, and POI. Other AMIA members joined in significant numbers. It immediately became a major forum for discussing medical informatics, informatics policies, and discussion of the purpose of AMIA itself. The listserv membership approximates 25% of AMIA's members and has generated over 6,000 posts. We report on a survey of the listserv's members: what members think about the listserv; what participants want for medical informatics; how they think those goals should be achieved, and what AMIA's role should be in this process. The listserv provides vital signs about AMIA and hopes for informatics. We combine qualitative analysis of members' comments and responses about the listserv using ATLAS.ti qualitative text analysis tool and a w...
Journal of the American Medical Informatics Association : JAMIA, 2015
In many hospitals and health systems, a 'new' electronic health record means a shift to o... more In many hospitals and health systems, a 'new' electronic health record means a shift to one vendor: Epic, a vendor that dominates in large and medium hospital markets and continues its success with smaller institutions and ambulatory practices. Our paper examines the implications of this emerging monoculture: its advantages and disadvantages for physicians and hospitals and its role in innovation, professional autonomy, implementation difficulties, workflow, flexibility, cost, data standards, interoperability, and interactions with other information technology (IT) systems.
IFIP International Federation for Information Processing
In medical education and clinical care, representations of the patient help health care teams in ... more In medical education and clinical care, representations of the patient help health care teams in planning and coordinating patient care, sometimes over geographic distances. This takes forms ranging from telemedicine consultations to using simulations and information and communication technology representations to plan, and at times, perform clinical procedures such as are done in intensive care units or in surgery. The increasing reliance on computer-mediated interaction in health care generally is considered the means to more efficient, equitable, and cost-effective care with reduced errors. Clinical work, then, may be carried out with simulated images and processes rather than through such physical processes as examining the patient directly. Instead of treating the actual person, one result may be that clinicians are treating computer-mediated representations of that person.
Uploads
Papers by Ross Koppel