Advancing Validity in Outcome Evaluation: Theory and Practice: New Directions for Evaluation, Number 130

ISBN-13: 9781118094075 / Angielski / Miękka / 2011 / 136 str.

Huey T. Chen;Stewart I. Donaldson;Melvin M. Mark

Opis

Exploring the influence and application of Campbellian validity typology in the theory and practice of outcome evaluation, this volume addresses the strengths and weaknesses of this often controversial evaluation method and presents new perspectives for its use. Editors Huey T. Chen, Stewart I. Donaldson and Melvin M. Mark provide a historical overview of the Campbellian typology adoption, contributions and criticism. Contributing authors propose strategies in developing a new perspective of validity typology for advancing validity in program evaluation including

1. Enhance External Validity
2. Enhance Precision by Reclassifying the Campbellian Typology
3. Expand the Scope of the Typology

The volume concludes with William R. Shadish's spirited rebuttal to earlier chapters. A collaborator with Don Campbell, Shadish provides a balance to the perspective of the issue with a clarification and defense of Campbell's work. This is the 129th volume of the Jossey-Bass quarterly report series New Directions for Evaluation, an official publication of the American Evaluation Association.

Spis treści

EDITORS NOTES 1
Huey T. Chen, Stewart I. Donaldson, Melvin M. Mark

1. Validity Frameworks for Outcome Evaluation 5
Huey T. Chen, Stewart I. Donaldson, Melvin M. Mark

This chapter discusses the concept of validity as it applies to outcome evaluation and the contributions of the Campbellian validity typology, as well as related criticisms, and overviews the issue.

2. What Works for Whom, Where, Why, for What, and When? Using Evaluation Evidence to Take Action in Local Contexts 17
John Gargani, Stewart I. Donaldson

After discussing limits of the Campbellian tradition regarding external validity, this chapter argues that the external validity of an evaluation could be enhanced by better addressing issues about what works for whom, where, why, for what, and when.

3. New (and Old) Directions for Validity Concerning Generalizability 31
Melvin M. Mark

This chapter reviews several alternative framings of generalizability issues and provides potentially fruitful directions for enhancing external validity in outcome evaluation.

4. Criticisms of and an Alternative to the Shadish, Cook, and Campbell Validity Typology 43
Charles S. Reichardt

This chapter presents four criticisms of the Shadish, Cook, and Campbell (2002) typology of validity. An alternative typology is proposed that avoids these criticisms.

5. Reframing Validity in Research and Evaluation: A Multidimensional, Systematic Model of Valid Inference 55
George Julnes

A validity framework is described with three dimensions representation (construct validity), causal inference (internal and external validity), and valuation.

6. Conflict of Interest and Campbellian Validity 69
Ernest R. House

Problems related to bias due to researchers intentional and unintentional manipulation are discussed, as are strategies for dealing with such problems and how they might be incorporated within the Campbellian validity tradition.

7. The Construct(ion) of Validity as Argument 81
Jennifer C. Greene

This chapter presents an interpretive/constructivist perspective on outcome evaluation and on the warrants for our outcome–evaluation conclusions. It underscores the importance of developing warrants through argumentation, in addition to selected empirical evidence.

8. Assessing Program Outcomes From the Bottom–Up Approach: An Innovative Perspective to Outcome Evaluation 93
Huey T. Chen, Paul Garbe

The authors argue that, to be stakeholder responsive, evaluation must apply an integrative validity model and a bottom–up approach to outcome evaluation to address both scientific and practical issues.

9. The Truth About Validity 107
William R. Shadish

This chapter discusses the contribution of the chapters in the issue, including the extent to which they offer something new and provide justified arguments, and considers how discussions of validity might contribute productively to evaluation theory and practice.

INDEX 119

Blurb

From the Editors

Like many evaluators, we have applied the Campbellian validity typology to outcome evaluations, even though the typology was developed for general research purposes rather than for evaluation specifically. The application to program evaluation has created mixed results. On the one hand, when designing a rigorous outcome evaluation we greatly benefit from the principles and methods highlighted by the typology. The typology enhances our abilities to present or defend the evidence provided from an outcome evaluation. On the other hand, we find that evaluation results often do not meet stakeholders′ needs and expectations. The mixed experiences puzzled us, but also motivated us to seek possible solutions as represented in this issue. In general, we take the stance that revising or expanding the Campbellian typology can advance validity in outcome evaluation. Chapter authors present multiple views on how to build on the Campbellian typology′s contribution or suggest alternative validity frameworks or models to serve program evaluation better. We hope these perspectives advance theory and practice regarding validity in evaluation, as well as improve the quality and usefulness of outcome evaluations.