Mismeasuring Impact

Demonstrating the effectiveness of nonprofit social programs is a challenge. This has led to a rapid rise in nonprofit organizations' use of randomized controlled trials (RCTs), for evaluation. As a result, most nonprofit sector professionals today can tell you why nonprofits should do an RCT. This book tells you why they probably shouldn't. Mismeasuring Impact explores why RCTs are being embraced as the "gold standard" for nonprofit evaluation, despite the high cost and time investment required, and the problems with using RCTs in a nonprofit context. The book describes what happens inside nonprofits when they take part in RCTs, the unintended equity issues that arise, and why nonprofits decide to participate in RCTs despite the problems. University of Chicago professors Marwell and Mosley's research is based on extensive interviews with key players: nonprofit managers, professional program evaluators, and program officers in philanthropic foundations. The book explains why most RCTs conducted in nonprofits fail to meet required standards for rigor, undercutting their accuracy. Utlimately, RCTs are used to create nonprofit legitimacy, not to foster nonprofit improvement. RCTs also privilege program and organizational standardization over the key strengths of nonprofit organizations: innovation and responsiveness to community needs. Nonprofits and funders need forms of evaluation that lift up these strengths, but RCTs fall short. Mismeasuring Impact offers alternatives that build strong organizations, not just standardized programs. The book concludes by suggesting evaluation approaches that funders and nonprofits of all sizes can support.