Skip to main content



Child-Maltreatment-Research-L (CMRL) List Serve

Database of Past CMRL Messages

Welcome to the database of past Child-Maltreatment-Research-L (CMRL) list serve messages. The table below contains all past CMRL messages (text only, no attachments) from Nov. 20, 1996 - March 6, 2018 and is updated quarterly.

Instructions: Postings are listed for browsing with the newest messages first. Click on the linked ID number to see a message. You can search the author, subject, message ID, and message content fields by entering your criteria into this search box:

Message ID: 7986
Date: 2009-01-05

Author:Don Cohon

Subject:FW: Evidence-based and "Mix and Match" Programs

Forwarding comments on this thread by the Director of Programs and Strategy

at my agency.



Don Cohon, Ph.D., Director

Institute for the Study of Community-Based Services

Edgewood Center for Children and Families

1801 Vicente Street

San Francisco, CA 94116



(415) 383-2273





-----Original Message-----

From: Ken Epstein [

Sent: Tuesday, December 30, 2008 11:58 PM

To: Don Cohon--UCSF

Subject: FW: Evidence-based and "Mix and Match" Programs



Yes this is a good discussion but it needs to have an interface with

Program people. I fear that the conversation does not address the

realities of implementing programs in the fiscal and overlitigious

environment we are in and that it does not mention the costs associated

with buying the components or the manualized practice, having a research

group do the research, having supervisors and mentors, maintaining staff

and building sustainable funding for programs. Our funders more and

more want us to provide evidence based programs and less and less give

us the dollars and time to build these processes. I also agree

wholeheartedly with the notion that more is not good. However I disagree

that we need to forego addressing poverty or racism or poor education as

a source. If we continue to support environments that increasingly

expose our youth and adults to more and more violence, then we create

more and more candidates for programs with less and less resources.

Simply put violent crime is down in the US except with African American

Teens where it has increased dramatically.







I fear that the evidence revolution will ultimately pick and choose good

candidates for success with limited interventions that are tested but

cannot be widely replicated or sustained and are not effective with the

most neediest and most challenging families.





-----Original Message-----

From: bounce-3426746-6833904@list.cornell.edu

[mailto:bounce-3426746-6833904@list.cornell.edu] On Behalf Of Ben

Saunders

Sent: Monday, December 29, 2008 10:20 AM

To: Child Maltreatment Researchers

Subject: Re: Evidence-based and "Mix and Match" Programs







Sandra,



Absolutely agree with your comments (other than the "titan"



characterization). Without question, neighborhood and other contextual



factors have an enormous impact not only on the functioning of people



living in them, but also on the range of options available to helping



professionals trying to do their job. Our challenge is, how can we best

do good child welfare work in very bad contexts with families with



multiple problems. Do we have to cure poverty, crime, unemployment,



poor education, housing shortages, and discrimination in order to do



quality child welfare work? I hope not. But do these social ills make



it tough work? Absolutely. I think this is the focus part Mark is



speaking about and I am agreeing with. (Mark, I am sure you will feel



free to correct me.) It may be that in trying to mitigate all problems



with multiple, specific interventions/programs we actually decrease the



potential for the positive impact of any of them. Giving direction for



the most effective mix of interventions with which families is one of



the jobs of research. The good news is that we seem to be learning more

about this. It may be that by actually doing one or two things that



effectively attack a limited set of problems, we will have a more



positive impact not only on the targeted problems but also on many other

problems compared to trying to do a lot of things all at once. The



burden of multiple interventions imposed on families (and workers) may



reduce their individual effectiveness, perhaps because they are never



properly delivered. There also may be negative interactions between



interventions that inherently reduce their impact even when families



participate in everything assigned. These are all hypotheses to be



tested. However, the evidence does seem to be mounting that there is



not simply a linear, additive effect to interventions so that more is



better. A primary challenge for researchers right now probably is not



to develop and test yet more interventions and programs. Rather, we



might want to examine the "reach" or generalization effects to



nontargeted problems of some of our more effective existing



interventions and the interactions positive or negative, between them.



Mark's work with PCIT is a great example of that sort of work. Without



this sort of research to base their work on, I think you are right,



frontline workers will continue to see awful situations and think trying

to give the family everything is the way to go. Without reliable



information to the contrary, why would they not? As you say, it feels



right to do so and the stakes often are very high.







I also agree that any system that does not provide effective training,



ongoing supervision, and ready access to useful information and practice

tools for its front-line workforce is unlikely to produce good work, and

the child welfare and mental health systems are no different. Our



challenge as researchers is to figure out better ways of doing that



effectively and efficiently. The emerging dissemination and



implementation research literature can be of help here, as well as the



substantial dissemination work going on around the country. I am



optimistic. In our statewide EBT dissemination and implementation



project (Project BEST) we find that the practice wisdom of frontline



caseworkers is not contrary to the existing research, as my training



example in the previous post illustrated. Caseworkers have a pretty



good idea which programs work and which do not. They lament that



ineffective programs are often part of the "standard" treatment plan



institutionalized on the agency computer system and that effective,



evidence-based programs often are not available in their community.



They know treatment plans with multiple interventions often fail even



with highly motivated families because they are unrealistic for the



context. After relatively brief training, they are excited about the



notion of evidence-based treatment planning, and for the most part have



a good feel for titrating interventions and capitalizing on the



strengths of families in treatment plans. With ongoing consultation,



practice tools, and administrative support, they are able to put it in



practice. Our state is rethinking its traditional approach to new



worker training as a result. However, more research is needed to better

understand optimal approaches to training, implementation and



sustainability. How best to get what is known incorporated into the day

to day work of the frontline worker is yet another challenge for



researchers. And this is a tough one.







Ben







Sandra Azar wrote:



> Mark and Ben,



>



> I am hesitant to get into the dialogue of two such titans in the

field, but



> am struck as I always am about the missing elements in our treatment



> discussions. Both of you make incredibly important points and I too

believe



> deeply in science guiding our planning of treatment. But this

discussion



> leaves out two important elements in child protection work -- that I

believe



> we often we as researchers ignore -- WHO is actually doing the work

and the



> CONTEXT in which it is done. Coupled with the risk of a child death,

both



> may explain the desire to add and add and add.



>



> Who - Most parent home visitors and caseworkers are not trained to do

the



> kind of work we want them to do. Indeed, I have seen home visitors

who are



> bank tellers or human resource workers or ones who did housing work.

None



> with prior training more than a workshop here and there. So we can

develop



> whatever intervention we like --we are still expecting someone to

carry it



> out. In many contract agency settings, supervision is minimal and



> backgrounds of staff do not match the requirements of our protocols.



>



> Where - In urban settings when I have visited homes to conduct

research



> interviews or in the past to do home interventions, I often have gone

to



> neighborhoods where a shooting has taken place just a week ago

(setting off



> a sense of vigilance even though I am seasoned in going to such



> neighborhoods). I might have to step over a drunken neighbor who is

passed



> out on the stoop or the half drunk one, who insists on having a

conversation



> with me before I can get past them to get into the home. I then enter

homes



> where at times I am treated to a shouting argument between a man and a

woman



> that at moment feels like it will erupt into a physical fight. Other

men



> enter the home during the visit and stare rather belligerently at me

or join



> in taking the boyfriend's or husband's side. The woman may have no

food and



> the child appears sick. She is crying. This is the context where the

work



> is being done. Cognitive science tells us that cognition is disrupted

under



> such conditions.



>



> Put a frightened young worker and this context together and add high

risk



> for a child and the worker is overwhelmed and wants all the help she

can get



> and probably is shaking too much to carry out anything in any

standardized



> way. It FEELS like more is better -- because the situation has so many



> elements. She may habituate to this over time and with good

supervision grow



> into a decent practitioner who can carry out focused approaches. More

likely



> if she is good, she will leave the work for a job in a setting where

she can



> feel safe and carry out her work more effectively.



>



> Yes -- in a the context of a workshop and under ideal conditions with

the



> right coach, caseworkers can come up with a good treatment plan that

is



> focused and may be enough to produce results -- but that is with a

coach



> like one of you and the safety of sitting in a room with nothing to

lose and



> no threats hanging over them.



>



> Treatment development work in child maltreatment is unlike other

treatment



> development areas where most interventions are carried out by

professionals



> who have much education and in the context of an office setting. I

want the



> field to develop science based protocols, but until we deal with

staffing



> issues and focus our attention on methods for coping for staff -- the

"add



> and add and add" approach will continue to feel safer for most.



>



> Sandra



>



> ------------



> Sandra T. Azar, Ph.D.



> Professor



> Psychology Department, Moore 541



> Pennsylvania State University



> University Park, PA l6801



> 814-863-6019 (office)



> sta10@psu.edu



>



> -----Original Message-----



> From: bounce-3424753-6833833@list.cornell.edu



> [mailto:bounce-3424753-6833833@list.cornell.edu] On Behalf Of Saunders

PhD,



> Benjamin E



> Sent: Friday, December 26, 2008 11:44 PM



> To: Child Maltreatment Researchers



> Subject: RE: Evidence-based and "Mix and Match" Programs



>



> Mark,



> Thank you for the excellent summary of several very important issues.

Two



> points are particularly critical for future research. IMHO, the

current



> excitement in some quarters over "components" approaches to treatment

vs.



> manualized "protocols" is, from a research perspective, a red herring.

As



> you note, both must have some sort of decision rules about what to do

next



> at certain points in treatment. If not, they just become virtually

random



> in nature. So, the results of those decisions will need to be tested



> empirically whether it means following a "protocol" or decision rules

about



> using components. Frankly, when one scratches the surface, the two



> approaches sound suspiciously similar.



>



> Some have suggested that components approaches are more efficient

because



> they use only the "active ingredient" components of protocols at key

points



> in the treatment and skip the unnecessary stuff. Unfortunately there

is



> precious little dismantling research discerning exactly what those

active



> ingredients are, and whether or not they only get active when the

other



> "unnecessary" components have been used as well (what one might call



> conditional component efficacy). It may turn out that the components

that



> have been picked to be used actually do have the most impact even when

not



> used in concert with other techniques. Or not. This hypothesis

remains to



> be tested for most approaches.



>



> Others have suggested that "components" approaches are more palpable

to



> clinicians because then they can use their clinical judgment when to

do what



> rather than following the strict rules of a protocol. However, as you

point



> out, many of the components approaches then proceed to teach elaborate

rules



> for when and how to use the particular components chosen and end up

being



> more complicated than protocols (but without the outcome research to

support



> their efficacy). While the whole components vs. protocols debate is

an



> interesting pastime for some of us, from an empirical testing

standpoint, it



> may be a debate without a difference. The empirical question still

is,



> "When therapists do this, do clients get better compared to when

therapists



> do that?" Call it what you will, components, protocols, or whatever,

you



> still have to define the "this" (aka independent variable) in

sufficiently



> replicable manner.



>



> You second point about "focused" vs. "comprehensive" treatment

planning also



> is absolutely critical. At a recent training for about 50 CPS

workers, I



> gave them all a case we had seen recently in our clinic and asked them

to



> break into groups and come up with a treatment plan. the case was a

typical



> train wreck, multiproblem, abusive family. We then wrote on a flip

chart



> all of the interventions, treatments, programs and meetings they

thought the



> family should receive and go to. It took 4 large flip chart sheets to

write



> them all down. I then asked the workers two questions. First, did

they



> think any family in the world, much less this family, could get their

child



> to even half of the appointments they were recommending. Second, did

anyone



> in the room believe this was an effective treatment plan that would



> accomplish the goals we had set for this family. No one in the room



> believed any family (even their own) could accomplish half of the

treatment



> plan, and not one person thought it was an effective treatment plan.

Yet



> they wrote it. They agreed that they have been trained and

acculturated to



> simply add and add and add and add to treatment plans to the point of

being



> ridiculous. The good news is that at the end of the day, when

challenged to



> come up with a feasible treatment plan composed of evidence supported



> interventions and programs, they were able to do it and the plan was

about



> 1/3 of a flip chart page.



>



> The notions that more is better and that doing an untested something

is



> always better than doing nothing permeates the system and needs to be



> challenged.



>



> Again, thanks for the elegant thoughts.



>



> Ben



>



> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



> Benjamin E. Saunders, Ph.D.



> National Crime Victims Research and Treatment Center



> Medical University of South Carolina 843-792-2945 Phone



> Charleston, SC 29425 843-792-7146

Fax



>



> Visit our web sites: www.musc.edu/ncvc www.musc.edu/tfcbt



> www.musc.edu/ctg

www.musc.edu/saprevention



> ________________________________________



> From: bounce-3422423-6832002@list.cornell.edu



> [bounce-3422423-6832002@list.cornell.edu] On Behalf Of Chaffin, Mark

J.



> (HSC) [Mark-Chaffin@ouhsc.edu]



> Sent: Wednesday, December 24, 2008 8:31 PM



> To: Child Maltreatment Researchers



> Subject: RE: Evidence-based and "Mix and Match" Programs



>



> Tom,



>



> Yaiiii....where to start. There is considerable interest in the idea

of



> extracting common elements from across evidence based models, then

applying



> these depending on assessed case characteristics and some systematic



> algorithm. Probably the most detailed system of how this process

might be



> undertaken has been described by Chorpita and colleagues. Note,

however,



> that the processes for identifying both the elements themselves and

the



> matching algorithm as described by Chorpita are NOT just a matter of



> logic-model 'mix and match' eyeballing based on the clinician's gut or



> personal preferences. It is a quite structured and quantitative

process.



> It is still a protocol (and a complicated one at that), and not at all

the



> same thing as free-styling, fly-by-the-seat-of-the-pants services

often



> advocated by the anti-EBT crowd. If anything, it is a system probably



> requiring more expertise and training than most simple EBT's. How

well



> these algorithm-driven, elements-based systems work in practice is a

subject



> of ongoing study. The jury is not in. Some EBT's such as MST, have

done



> essentially this same thing for years and its worked out well. MST,

for



> example, is not a single protocol, but is assessment driven and

tailored,



> yet focuses the intervention on what we know matters, using basic

elements



> that are known to work In this sense, there are fairly complex EBT's,

such



> as MST, which target broad populations, and more specific EBT's, such

as



> TF-CBT for example, which are specific for particular types of

well-defined



> problems (i.e. PTSD).



>



> The elements-based idea does have a clear appeal--EBT's are probably

not



> monolithic entities incapable of being subdivided. Most share common



> elements within a given domain (e.g. most evidence-based parenting

programs



> share many common elements such as use of labeled praise, application

of



> structured time-out protocols, etc.). However, I would be very

skeptical of



> efforts to use this rationale to neuter EBT's, or to simply say "oh,

we're



> already doing all that" or "this is the same thing" when it really

isn't the



> case. One need look no further than the Blueprints implementations to

find



> evidence of this, where well-intended shooting from the hip led to

blending



> in ad hoc crap with EBT's and spoiled the results. The point is that

the



> elements-based approach is not an excuse to allow anything and

everything to



> come into the intervention or to mix-and-match without some fairly

tight



> limits, such as those described by Chorpita and colleagues.



>



> The area of most concern in your question lies in the idea of whether

more



> is better. There is excellent evidence at this point that this is not

only



> false but that more can become harmful. For example, where parenting



> interventions are concerned, it appears that adding additional

services to a



> parenting program actually poisons the benefits (Kaminsky, et al.



> meta-analysis). CPS and courts are notorious for this

misconception--often



> prescribing so many services that whatever benefits any of the

services



> might have offered may be quickly lost in the confusion. The analogy

to



> "polypharmacy" in psychiatry is not a bad one. Focus (i.e. directly



> behaviorally targeting the top priority that needs to change) rather

than



> comprehensiveness (trying to fix everything), is the new watchword in

many



> service systems. Exactly how much is too much, and what demands the



> highest priority is an unanswered empirical question. But the

emerging



> science in this area does, IMO, suggest a couple of general

principles.



> First, tailoring might work...IF the elements are selected carefully

based



> on scientific evidence and are clear essential common components

across



> EBT's. Second, there are clearly both practical and therapeutic

limits to



> how many things can be done well at once, and this point is reached

rapidly



> and past that point quickly begins to ruin the overall service

benefit. So,



> "focus" not "comprehensive" should be the watchword. We need to

emphasize



> this watchword because practitioners have been so imbued with the idea

that



> "comprehensive" is necessary that it takes considerable effort to

disabuse



> them of this unfortunate misconception. This is another reason why

any



> novel algorithm-driven, elements-based protocols need to be

structured. And



> why it needs to be rigorously evaluated. Keep in mind that

implementations



> of novel blended or elements-based programs cannot properly be called



> evidence-based just because the sources for the elements were



> evidence-based. For that matter, it cannot even be presumed to be

effective,



> altough we might predict that it would be.



>



> Mark



>



>



> ________________________________________



> From: Tom Hanna [tph3@cornell.edu]



> Sent: Thursday, December 18, 2008 8:25 AM



> Subject: Evidence-based and "Mix and Match" Programs



>



> On another list, there is active discussion underway on starting an



> "ancillary" parenting education program to an existing "core" home



> visitation program. The conversation quickly turned to the topic of



> "evidence-based", and then to funders and their requirements.



>



> The picture quickly got cloudy for me:



>



> 1. Some folks who already have an ancillary parenting education



> program reported that "blending" aspects of two evidence-based



> programs allowed them to tailor the trainings to the specific needs



> of their "home visited" parents. Others quickly pointed out that



> this is "wrong" and should not be done -- neither evidence based



> program is being followed precisely, and therefore both are



> "contaminated." Funders frown.



>



> 2. No one has said what additive effect, if any, is expected from



> providing a parenting program on top of a home visitation program.



> The underlying assumption is that families will be better off with



> two distinct interventions instead of one. (In fact, many centers in



> this home visiting network have many ancillary programs that serve



> some if not many of their home visited families.)



>



> 3. I know that lots of funders are demanding that agencies use



> "evidence-based" programs. But I now learn that funders are pushing



> implementation of a "matrix" of "evidence-based" programs. The



> underlying assumption is that "if one evidence-based program is good



> for families, then many are better."



>



> My question: Is there any research that helps multi-service agencies



> make their way through this minefield when working with a cohort of



> families?

<

Forwarding comments on this thread by the Director of Programs and Strategy

at my agency.



Don Cohon, Ph.D., Director

Institute for the Study of Community-Based Services

Edgewood Center for Children and Families

1801 Vicente Street

San Francisco, CA 94116



(415) 383-2273





-----Original Message-----

From: Ken Epstein [

Sent: Tuesday, December 30, 2008 11:58 PM

To: Don Cohon--UCSF

Subject: FW: Evidence-based and "Mix and Match" Programs



Yes this is a good discussion but it needs to have an interface with

Program people. I fear that the conversation does not address the

realities of implementing programs in the fiscal and overlitigious

environment we are in and that it does not mention the costs associated

with buying the components or the manualized practice, having a research

group do the research, having supervisors and mentors, maintaining staff

and building sustainable funding for programs. Our funders more and

more want us to provide evidence based programs and less and less give

us the dollars and time to build these processes. I also agree

wholeheartedly with the notion that more is not good. However I disagree

that we need to forego addressing poverty or racism or poor education as

a source. If we continue to support environments that increasingly

expose our youth and adults to more and more violence, then we create

more and more candidates for programs with less and less resources.

Simply put violent crime is down in the US except with African American

Teens where it has increased dramatically.







I fear that the evidence revolution will ultimately pick and choose good

candidates for success with limited interventions that are tested but

cannot be widely replicated or sustained and are not effective with the

most neediest and most challenging families.





-----Original Message-----

From: bounce-3426746-6833904list.cornell.edu

[mailto:bounce-3426746-6833904list.cornell.edu] On Behalf Of Ben

Saunders

Sent: Monday, December 29, 2008 10:20 AM

To: Child Maltreatment Researchers

Subject: Re: Evidence-based and "Mix and Match" Programs







Sandra,



Absolutely agree with your comments (other than the "titan"



characterization). Without question, neighborhood and other contextual



factors have an enormous impact not only on the functioning of people



living in them, but also on the range of options available to helping



professionals trying to do their job. Our challenge is, how can we best

do good child welfare work in very bad contexts with families with



multiple problems. Do we have to cure poverty, crime, unemployment,



poor education, housing shortages, and discrimination in order to do



quality child welfare work? I hope not. But do these social ills make



it tough work? Absolutely. I think this is the focus part Mark is



speaking about and I am agreeing with. (Mark, I am sure you will feel



free to correct me.) It may be that in trying to mitigate all problems



with multiple, specific interventions/programs we actually decrease the



potential for the positive impact of any of them. Giving direction for



the most effective mix of interventions with which families is one of



the jobs of research. The good news is that we seem to be learning more

about this. It may be that by actually doing one or two things that



effectively attack a limited set of problems, we will have a more



positive impact not only on the targeted problems but also on many other

problems compared to trying to do a lot of things all at once. The



burden of multiple interventions imposed on families (and workers) may



reduce their individual effectiveness, perhaps because they are never



properly delivered. There also may be negative interactions between



interventions that inherently reduce their impact even when families



participate in everything assigned. These are all hypotheses to be



tested. However, the evidence does seem to be mounting that there is



not simply a linear, additive effect to interventions so that more is



better. A primary challenge for researchers right now probably is not



to develop and test yet more interventions and programs. Rather, we



might want to examine the "reach" or generalization effects to



nontargeted problems of some of our more effective existing



interventions and the interactions positive or negative, between them.



Mark's work with PCIT is a great example of that sort of work. Without



this sort of research to base their work on, I think you are right,



frontline workers will continue to see awful situations and think trying

to give the family everything is the way to go. Without reliable



information to the contrary, why would they not? As you say, it feels



right to do so and the stakes often are very high.







I also agree that any system that does not provide effective training,



ongoing supervision, and ready access to useful information and practice

tools for its front-line workforce is unlikely to produce good work, and

the child welfare and mental health systems are no different. Our



challenge as researchers is to figure out better ways of doing that



effectively and efficiently. The emerging dissemination and



implementation research literature can be of help here, as well as the



substantial dissemination work going on around the country. I am



optimistic. In our statewide EBT dissemination and implementation



project (Project BEST) we find that the practice wisdom of frontline



caseworkers is not contrary to the existing research, as my training



example in the previous post illustrated. Caseworkers have a pretty



good idea which programs work and which do not. They lament that



ineffective programs are often part of the "standard" treatment plan



institutionalized on the agency computer system and that effective,



evidence-based programs often are not available in their community.



They know treatment plans with multiple interventions often fail even



with highly motivated families because they are unrealistic for the



context. After relatively brief training, they are excited about the



notion of evidence-based treatment planning, and for the most part have



a good feel for titrating interventions and capitalizing on the



strengths of families in treatment plans. With ongoing consultation,



practice tools, and administrative support, they are able to put it in



practice. Our state is rethinking its traditional approach to new



worker training as a result. However, more research is needed to better

understand optimal approaches to training, implementation and



sustainability. How best to get what is known incorporated into the day

to day work of the frontline worker is yet another challenge for



researchers. And this is a tough one.







Ben







Sandra Azar wrote:



> Mark and Ben,



>



> I am hesitant to get into the dialogue of two such titans in the

field, but



> am struck as I always am about the missing elements in our treatment



> discussions. Both of you make incredibly important points and I too

believe



> deeply in science guiding our planning of treatment. But this

discussion



> leaves out two important elements in child protection work -- that I

believe



> we often we as researchers ignore -- WHO is actually doing the work

and the



> CONTEXT in which it is done. Coupled with the risk of a child death,

both



> may explain the desire to add and add and add.



>



> Who - Most parent home visitors and caseworkers are not trained to do

the



> kind of work we want them to do. Indeed, I have seen home visitors

who are



> bank tellers or human resource workers or ones who did housing work.

None



> with prior training more than a workshop here and there. So we can

develop



> whatever intervention we like --we are still expecting someone to

carry it



> out. In many contract agency settings, supervision is minimal and



> backgrounds of staff do not match the requirements of our protocols.



>



> Where - In urban settings when I have visited homes to conduct

research



> interviews or in the past to do home interventions, I often have gone

to



> neighborhoods where a shooting has taken place just a week ago

(setting off



> a sense of vigilance even though I am seasoned in going to such



> neighborhoods). I might have to step over a drunken neighbor who is

passed



> out on the stoop or the half drunk one, who insists on having a

conversation



> with me before I can get past them to get into the home. I then enter

homes



> where at times I am treated to a shouting argument between a man and a

woman



> that at moment feels like it will erupt into a physical fight. Other

men



> enter the home during the visit and stare rather belligerently at me

or join



> in taking the boyfriend's or husband's side. The woman may have no

food and



> the child appears sick. She is crying. This is the context where the

work



> is being done. Cognitive science tells us that cognition is disrupted

under



> such conditions.



>



> Put a frightened young worker and this context together and add high

risk



> for a child and the worker is overwhelmed and wants all the help she

can get



> and probably is shaking too much to carry out anything in any

standardized



> way. It FEELS like more is better -- because the situation has so many



> elements. She may habituate to this over time and with good

supervision grow



> into a decent practitioner who can carry out focused approaches. More

likely



> if she is good, she will leave the work for a job in a setting where

she can



> feel safe and carry out her work more effectively.



>



> Yes -- in a the context of a workshop and under ideal conditions with

the



> right coach, caseworkers can come up with a good treatment plan that

is



> focused and may be enough to produce results -- but that is with a

coach



> like one of you and the safety of sitting in a room with nothing to

lose and



> no threats hanging over them.



>



> Treatment development work in child maltreatment is unlike other

treatment



> development areas where most interventions are carried out by

professionals



> who have much education and in the context of an office setting. I

want the



> field to develop science based protocols, but until we deal with

staffing



> issues and focus our attention on methods for coping for staff -- the

"add



> and add and add" approach will continue to feel safer for most.



>



> Sandra



>



> ------------



> Sandra T. Azar, Ph.D.



> Professor



> Psychology Department, Moore 541



> Pennsylvania State University



> University Park, PA l6801



> 814-863-6019 (office)



> sta10psu.edu



>



> -----Original Message-----



> From: bounce-3424753-6833833list.cornell.edu



> [mailto:bounce-3424753-6833833list.cornell.edu] On Behalf Of Saunders

PhD,



> Benjamin E



> Sent: Friday, December 26, 2008 11:44 PM



> To: Child Maltreatment Researchers



> Subject: RE: Evidence-based and "Mix and Match" Programs



>



> Mark,



> Thank you for the excellent summary of several very important issues.

Two



> points are particularly critical for future research. IMHO, the

current



> excitement in some quarters over "components" approaches to treatment

vs.



> manualized "protocols" is, from a research perspective, a red herring.

As



> you note, both must have some sort of decision rules about what to do

next



> at certain points in treatment. If not, they just become virtually

random



> in nature. So, the results of those decisions will need to be tested



> empirically whether it means following a "protocol" or decision rules

about



> using components. Frankly, when one scratches the surface, the two



> approaches sound suspiciously similar.



>



> Some have suggested that components approaches are more efficient

because



> they use only the "active ingredient" components of protocols at key

points



> in the treatment and skip the unnecessary stuff. Unfortunately there

is



> precious little dismantling research discerning exactly what those

active



> ingredients are, and whether or not they only get active when the

other



> "unnecessary" components have been used as well (what one might call



> conditional component efficacy). It may turn out that the components

that



> have been picked to be used actually do have the most impact even when

not



> used in concert with other techniques. Or not. This hypothesis

remains to



> be tested for most approaches.



>



> Others have suggested that "components" approaches are more palpable

to



> clinicians because then they can use their clinical judgment when to

do what



> rather than following the strict rules of a protocol. However, as you

point



> out, many of the components approaches then proceed to teach elaborate

rules



> for when and how to use the particular components chosen and end up

being



> more complicated than protocols (but without the outcome research to

support



> their efficacy). While the whole components vs. protocols debate is

an



> interesting pastime for some of us, from an empirical testing

standpoint, it



> may be a debate without a difference. The empirical question still

is,



> "When therapists do this, do clients get better compared to when

therapists



> do that?" Call it what you will, components, protocols, or whatever,

you



> still have to define the "this" (aka independent variable) in

sufficiently



> replicable manner.



>



> You second point about "focused" vs. "comprehensive" treatment

planning also



> is absolutely critical. At a recent training for about 50 CPS

workers, I



> gave them all a case we had seen recently in our clinic and asked them

to



> break into groups and come up with a treatment plan. the case was a

typical



> train wreck, multiproblem, abusive family. We then wrote on a flip

chart



> all of the interventions, treatments, programs and meetings they

thought the



> family should receive and go to. It took 4 large flip chart sheets to

write



> them all down. I then asked the workers two questions. First, did

they



> think any family in the world, much less this family, could get their

child



> to even half of the appointments they were recommending. Second, did

anyone



> in the room believe this was an effective treatment plan that would



> accomplish the goals we had set for this family. No one in the room



> believed any family (even their own) could accomplish half of the

treatment



> plan, and not one person thought it was an effective treatment plan.

Yet



> they wrote it. They agreed that they have been trained and

acculturated to



> simply add and add and add and add to treatment plans to the point of

being



> ridiculous. The good news is that at the end of the day, when

challenged to



> come up with a feasible treatment plan composed of evidence supported



> interventions and programs, they were able to do it and the plan was

about



> 1/3 of a flip chart page.



>



> The notions that more is better and that doing an untested something

is



> always better than doing nothing permeates the system and needs to be



> challenged.



>



> Again, thanks for the elegant thoughts.



>



> Ben



>



> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



> Benjamin E. Saunders, Ph.D.



> National Crime Victims Research and Treatment Center



> Medical University of South Carolina 843-792-2945 Phone



> Charleston, SC 29425 843-792-7146

Fax



>



> Visit our web sites: www.musc.edu/ncvc www.musc.edu/tfcbt



> www.musc.edu/ctg

www.musc.edu/saprevention



> ________________________________________



> From: bounce-3422423-6832002list.cornell.edu



> [bounce-3422423-6832002list.cornell.edu] On Behalf Of Chaffin, Mark

J.



> (HSC) [Mark-Chaffinouhsc.edu]



> Sent: Wednesday, December 24, 2008 8:31 PM



> To: Child Maltreatment Researchers



> Subject: RE: Evidence-based and "Mix and Match" Programs



>



> Tom,



>



> Yaiiii....where to start. There is considerable interest in the idea

of



> extracting common elements from across evidence based models, then

applying



> these depending on assessed case characteristics and some systematic



> algorithm. Probably the most detailed system of how this process

might be



> undertaken has been described by Chorpita and colleagues. Note,

however,



> that the processes for identifying both the elements themselves and

the



> matching algorithm as described by Chorpita are NOT just a matter of



> logic-model 'mix and match' eyeballing based on the clinician's gut or



> personal preferences. It is a quite structured and quantitative

process.



> It is still a protocol (and a complicated one at that), and not at all

the



> same thing as free-styling, fly-by-the-seat-of-the-pants services

often



> advocated by the anti-EBT crowd. If anything, it is a system probably



> requiring more expertise and training than most simple EBT's. How

well



> these algorithm-driven, elements-based systems work in practice is a

subject



> of ongoing study. The jury is not in. Some EBT's such as MST, have

done



> essentially this same thing for years and its worked out well. MST,

for



> example, is not a single protocol, but is assessment driven and

tailored,



> yet focuses the intervention on what we know matters, using basic

elements



> that are known to work In this sense, there are fairly complex EBT's,

such



> as MST, which target broad populations, and more specific EBT's, such

as



> TF-CBT for example, which are specific for particular types of

well-defined



> problems (i.e. PTSD).



>



> The elements-based idea does have a clear appeal--EBT's are probably

not



> monolithic entities incapable of being subdivided. Most share common



> elements within a given domain (e.g. most evidence-based parenting

programs



> share many common elements such as use of labeled praise, application

of



> structured time-out protocols, etc.). However, I would be very

skeptical of



> efforts to use this rationale to neuter EBT's, or to simply say "oh,

we're



> already doing all that" or "this is the same thing" when it really

isn't the



> case. One need look no further than the Blueprints implementations to

find



> evidence of this, where well-intended shooting from the hip led to

blending



> in ad hoc crap with EBT's and spoiled the results. The point is that

the



> elements-based approach is not an excuse to allow anything and

everything to



> come into the intervention or to mix-and-match without some fairly

tight



> limits, such as those described by Chorpita and colleagues.



>



> The area of most concern in your question lies in the idea of whether

more



> is better. There is excellent evidence at this point that this is not

only



> false but that more can become harmful. For example, where parenting



> interventions are concerned, it appears that adding additional

services to a



> parenting program actually poisons the benefits (Kaminsky, et al.



> meta-analysis). CPS and courts are notorious for this

misconception--often



> prescribing so many services that whatever benefits any of the

services



> might have offered may be quickly lost in the confusion. The analogy

to



> "polypharmacy" in psychiatry is not a bad one. Focus (i.e. directly



> behaviorally targeting the top priority that needs to change) rather

than



> comprehensiveness (trying to fix everything), is the new watchword in

many



> service systems. Exactly how much is too much, and what demands the



> highest priority is an unanswered empirical question. But the

emerging



> science in this area does, IMO, suggest a couple of general

principles.



> First, tailoring might work...IF the elements are selected carefully

based



> on scientific evidence and are clear essential common components

across



> EBT's. Second, there are clearly both practical and therapeutic

limits to



> how many things can be done well at once, and this point is reached

rapidly



> and past that point quickly begins to ruin the overall service

benefit. So,



> "focus" not "comprehensive" should be the watchword. We need to

emphasize



> this watchword because practitioners have been so imbued with the idea

that



> "comprehensive" is necessary that it takes considerable effort to

disabuse



> them of this unfortunate misconception. This is another reason why

any



> novel algorithm-driven, elements-based protocols need to be

structured. And



> why it needs to be rigorously evaluated. Keep in mind that

implementations



> of novel blended or elements-based programs cannot properly be called



> evidence-based just because the sources for the elements were



> evidence-based. For that matter, it cannot even be presumed to be

effective,



> altough we might predict that it would be.



>



> Mark



>



>



> ________________________________________



> From: Tom Hanna [tph3cornell.edu]



> Sent: Thursday, December 18, 2008 8:25 AM



> Subject: Evidence-based and "Mix and Match" Programs



>



> On another list, there is active discussion underway on starting an



> "ancillary" parenting education program to an existing "core" home



> visitation program. The conversation quickly turned to the topic of



> "evidence-based", and then to funders and their requirements.



>



> The picture quickly got cloudy for me:



>



> 1. Some folks who already have an ancillary parenting education



> program reported that "blending" aspects of two evidence-based



> programs allowed them to tailor the trainings to the specific needs



> of their "home visited" parents. Others quickly pointed out that



> this is "wrong" and should not be done -- neither evidence based



> program is being followed precisely, and therefore both are



> "contaminated." Funders frown.



>



> 2. No one has said what additive effect, if any, is expected from



> providing a parenting program on top of a home visitation program.



> The underlying assumption is that families will be better off with



> two distinct interventions instead of one. (In fact, many centers in



> this home visiting network have many ancillary programs that serve



> some if not many of their home visited families.)



>



> 3. I know that lots of funders are demanding that agencies use



> "evidence-based" programs. But I now learn that funders are pushing



> implementation of a "matrix" of "evidence-based" programs. The



> underlying assumption is that "if one evidence-based program is good



> for families, then many are better."



>



> My question: Is there any research that helps multi-service agencies



> make their way through this minefield when working with a cohort of



> families?

<