Maltreatment Rate as an Outcome Indicator

["Mark J. Chaffin" <Mark-Chaffin@xxxxxxxxx>]

Sharon, 

Good questions.  We are currently conducting some similar evaluations of
programs for high-risk parents using future maltreatment reports as the
outcome of interest. Here are some thoughts.

* COMPARISONS  I would tend to agree with Walter that comparing maltreatment
rates of participants in programs for high-risk parents with those of the
general population is probably not a fair comparison.  But it is important
to compare them to something.  Absent experimental controls, etc., we're
found it useful to approach this from multiple angles, including comparing
program completers with dropouts (controlling initial risk levels if
possible), comparing program service recipients with comparable high-risk
recipients of much more limited services (or no services), and looking for
"dose" effects in the relationship between amount of services and outcomes,
controlling for level of severity. 

* WHO TO TRACK  We've found it preferable to track parents.  Reports on
children tend to inflate the rate (e.g. one parent's "event" can be logged
in the system as abuse and/or neglect for each of several kids, making it
look like multiple events, thus over-representing cases from larger
families). Also, if only children are tracked, the data will include
maltreatment by someone other than the parent who received the services
(e.g. sexual abuse by an extended family member, etc.).

* MONITORING OR SURVEILLANCE EFFECTS  We check all program "failures" to see
if the reporter was the service program or one of their staff.  This turned
out to be very rare, even for home-visiting programs.  Often, when the
person was reported by the program, there was also a report from another
independent source.  Overall, program-initiated reports uniquely accounted
for less than 4% of all failures.  Still, its a good idea to identify these.

* CONSENT  In our studies, we are required to include explicit consent to
follow-up through administrative data bases in the original consent form.  

* OTHER ISSUES  Because failure data is complicated by time, the simple
annual failure rate may not tell the entire story (i.e. some participants
may be followed for five years, others for a week).  Survival analysis,
multi-level models, or poisson models with number of events adjusted for
exposure might be worth considering. Also, the density of service delivery
may vary over time.  The bottom line is that its more revealing when you
include time in the analysis.  


Mark Chaffin
Center on Child Abuse and Neglect
Univ. of Oklahoma Health Sciences Center