Informing Federal Policies on Evaluation Methodology: Building the Evidence Base for Method Choice in Government Sponsored Evaluations

One of the most divisive issues in the evaluation community has been the debate over which methodologies are to be considered adequate or commendable in addressing different evaluation questions in different settings. One form of this debate involved opposing camps of proponents of qualitative versus quantitative methods. A decade ago, there was some hope that the two sides of this debate, referred to as the paradigm war, were learning to respect each other. More recently, however, a federal agency priority for funding random assignment experimental studies has reignited the debate. This volume provides a space for a productive dialogue that, by identifying areas of agreement but also fundamental differences, will promote a more durable working consensus on the circumstances in which some methods are to be preferred over others. The chapter authors and discussants make clear that there are different types of evidence with which to inform this dialogue, including empirical findings of the impact of method choice on evaluation outcomes, the evidence contained in the wisdom of practice, and the results of critical analyses of the broader social impacts of method choice. The editors build on these contributions to suggest pragmatic policies for federal agencies, promoting both context-appropriate method choice and the importance of managing portfolios of evaluative research that maintain desired distributions of methodologies. This is the 113th volume of the Jossey-Bass quarterly report series New Directions for Evaluation, a publication of Jossey-Bass and the American Evaluation Association.