“Moving From What Works To What Replicates:
A New Framework for Evidence-Based Decision-Making”

Vivian Wong
Assistant Professor of Research, Statistics, and Evaluation
University of Virginia

Monday, May 20th
12PM-1PM
Library Place 617

Given the central role of replication in the accumulation of scientific knowledge, researchers try to replicate seemingly well-established findings. Mounting evidence, however, suggests that results from many studies are fragile and hard to replicate. The so-called “replication crisis” has important implications for evidence-based decision-making in the health and social sciences. At the same time, there is intense debate about what constitutes a successful replication and why certain types of replication rates are so low. A crucial set of questions for evidence-based decision-making involve questions about external validity and replicability. We need to understand the contexts and conditions under which interventions produce similar outcomes. To address these challenges, I introduce a novel framework that provides a clear definition of replication, and highlights the conditions under which results are likely to replicate (Wong & Steiner, 2018). I present replication as a prospective research design. This makes it possible to define key assumptions for the direct replication of results, and to show how different replication designs can be derived and used to evaluate treatment effect heterogeneity. I argue that replication designs are feasible, desirable, and relevant in real world settings that are important for evidence-based decision-making.

Vivian C. Wong is a research methodologist in the field of Education. Currently, Dr. Wong is an Assistant Professor in Research, Statistics, and Evaluation in the Curry School of Education at the University of Virginia. Her research focuses on evaluating interventions in early childhood and K-12 systems. As a methodologist, her expertise is in improving the design, implementation and analysis of randomized experiments, regression-discontinuity, interrupted time series, and matching designs in field settings. She is currently examining the design, implementation, and analysis of replication studies in field settings, as well as developing innovative methods for evaluating No Child Left Behind. Dr. Wong participated in the Institute for Education Sciences (IES) Predoctoral Training Program at Northwestern University, and received the Outstanding IES Predoctoral Fellow Award in 2010 for her dissertation work on “Addressing Theoretical and Practical Challenges in the Regression-Discontinuity Design.” She is a Principal Member of IES’s Statistics and Methodology review panel.