Re: theory-based or theory-driven evaluations

[Anthony Petrosino <petrosan@xxxxxxxxxxxxxxxxxx>]

Shari,

You're not ignorant, there's much confusion in these terms.  Theory in this
case is confusing, since here it does not mean that the evaluation is
testing some explicit social science or scientific theory.  Instead, the
evaluation is testing the assumptions about how the program will work--an
evaluation of the "model" to be more exact.

I think most evaluations are based on some implicit program theory (not
necessarily social science or scientific theory), but it is usually a very
simple one (the program or policy will change an outcome).  For example,
many of the rehabilitation experiments in criminal justice that I have
collected have an underlying theory--the prevention program will reduce
offender recidivism.  But, as you can imagine, the actual mechanism by
which the program will reduce recidivism is probably a bit more complex
(program--leads to increased self-esteem and reduces juvenile idle
time--leads to less interaction with negative peers--leads to a reduction
in criminal behavior).

In a theory-based evaluation, policymakers and practitioners work with the
evaluator (hopefully) to identify these key assumptions or links, which we
might think of as a "model." These models can include programmatic as well
as mediating variables (and probably moderators as well, if one thought the
intervention varied along some moderator).  One would collect data on the
key "links"--along with the main outcome measure (e.g., recidivism in the
example above).

If anyone is interested, there is a burgeoning literature on this.  New
Directions in Program Evaluation published two entire volumes in 1987 and
1989 on theory-driven evaluations and program theory.  Chen and Rossi have
published their own volumes (1990, 1992 I believe).  Carol Weiss has a
chapter in her new book (1997) and several recent articles in Evaluation
Review and New Directions for Program Evaluation (1997).

At 03:42 PM 4/15/98 +0300, you wrote:
>
>Excuse my ignorance... 
>
>Maybe it's because I'm on the other side of the world and not really up on
>the latest jargon...
>
>but what is theory-based or theory-driven evaluations???
>Are not all evaluation based on theory?
>Can someone please enlighten me?
>
>Shari Ben Natan
>
>At 09:40 23/02/98 -0800, you wrote:
>>Dear Child-Maltreatment-Research-L members:
>>
>>The Harvard Project on Schooling & Children is trying to obtain reports of
>>"theory-based or theory-driven" evaluations.  
>>
>>In these kind of evaluations, there is a clear theory about why the program
>>is supposed to work on the outcome measure of interest.  Usually this
>>includes an analysis of both intermediate and outcome measures.
>>Intermediate measures are those that measure the crucial linkages in the
>>pathway to the outcomes.
>>
>>For example, a program theorizes that to impact delinquency, it must change
>>the "self-concept" of juveniles.  A theory-based evaluation of this
>>program, would at minimum, include measures of self-concept and
>delinquency.  
>>
>>If anyone has conducted or knows about a theory-based or theory-driven
>>evaluation, we would love to acquire the reference/citation.  We have
>>covered the electronic data bases (NCJRS, CJ Abstracts, Sociofile,
>>Psychinfo, Eric, Medline, NCCAN, etc.), but our search terms may have
>>missed important references.
>>
>>Thank you in advance for any help you may provide.
>>
>>Sincerely,
>>
>>Anthony
>>__________
>>
>>____________
>>
>>Anthony J. Petrosino 
>>Spencer Post-Doctoral Fellow in Evaluation
>>Harvard Project on Schooling & Children
>>126 Mt. Auburn Street
>>Cambridge, MA 02138
>>
>>Phone:617-496-6882
>>Fax:  617-495-1994
>>Email:Petrosan@hugse1.harvard.edu
>>
>>
>
>
____________

Anthony J. Petrosino 
Post-Doctoral Program in Evaluation
Harvard University
Harvard Project on Schooling & Children
126 Mt. Auburn Street
Cambridge, MA 02138

Phone:617-496-6882
Fax:  617-495-1994
Email:Petrosan@hugse1.harvard.edu