RE: structured decision-making

Hello,

Sheri raises good questions. No matter how good a tool is, it’s only as good as its use in the field. I’d love to report on a recently completed study that addresses part of this question, but the jurisdiction has not approved it for release just yet. It will show that fidelity can be improved by involving supervisors in a comparative case reading approach. More importantly, improved fidelity leads to improved outcomes.

There are some ways we look at fidelity administratively. The most crude way is simply to measure the extent to which assessments are being completed. We see implementation as a developmental process, with simply getting into the habit of completing the correct assessments at the correct time is a necessary, and hopefully brief, first step. Using aggregate data to identify patterns, it is possible to notice where it may be necessary to provide refresher training, or identify other barriers to completion and develop solutions.

More importantly, the next step is to check quality of assessments. Supervision is vital to this function. When key decisions are being discussed, the appropriate decision tool needs to be part of the discussion. We strongly encourage adopting a systematic approach to supervisory case reading. A small random sample of records is reviewed and rated on key measures of fidelity. Perhaps most important is the question of whether there is concurrence between the completed tool and the narrative. The test is whether the supervisor, having only the narrative to rely on, would have completed the tool exactly the same.

Finally, the ultimate fidelity question is whether the results of the assessment are shaping practice. This can be observed on a case level basis by the final question in the supervisory case reading review. For example, if a risk assessment result was very high risk, was a case opened? If it was low risk, was it NOT opened? When priority needs are identified, does the case plan address them? This fidelity question can also be observed using aggregate data. What % of low, moderate, high and very high risk cases were opened? Are higher risk families receiving more worker contact?

We do ask these questions during technical assistance visits and using management analytics, including SafeMeasures, but this has not been a formal study. There is no doubt in my mind that our field needs to focus on implementation of good models, whether SDM or other evidence-based practices. Research in this area would contribute significantly. Thanks for raising the question.

Raelene Freitag, MSW, Ph.D.

Director

Children's Research Center

426 S. Yellowstone Dr. #250

Madison, WI 53719

608-831-1180

www.nccd-crc.org

Please do not send any identifying or confidential information (such as names, birthdates, social security numbers) via e-mail. It is possible for third parties to intercept information transmitted in an e-mail. Identification numbers (such as case or referral numbers) may be included where necessary. Intercepting persons cannot use these numbers to identify a client unless they have access to the host application or database.

From: bounce-1609119-6833790@xxxxxxxxxxxxxxxx [mailto:bounce-1609119-6833790@xxxxxxxxxxxxxxxx] On Behalf Of D F MCMAHON
Sent: Wednesday, July 18, 2007 9:30 AM
To: Child Maltreatment Researchers
Subject: structured decision-making

I''d like to know about studies of the fidelity with which structured decision-making is implemented by caseworkers, to what extent training in the decision-making method affects fidelity, and to what extent the "extraordinary pressure" on caseworkers who witness situations "beyond the scope of typical human experience" impacts their reliance on structured decision-making. Or is "nuance" (of the individual case) as experienced by the caseworker the driving force in decision-making?

Also, the same question in regard to lay GALs.

Sheri McMahon