RE: Pediatricians Alerted to the Dangers ...--Also, the scientific prospects for safety assessment in general.

["Edward Pabon" <institute@xxxxxxxxxxxx>]

The issue of low rate is very important in looking at "data" to guide
practice. In addition, the issue of "definition" is also very important
in assessing the "data" for practice usage. "Unrelated" adult members is
very prevalent in minority families as natural supports or extended
family members. As a result, practitioners and researchers have to be
concerned as to impose their own definition of "family."

Dr. Ed. Pabon
JBInstitute

-----Original Message-----
From: owner-CHILD-MALTREATMENT-RESEARCH-L@cornell.edu
[mailto:owner-CHILD-MALTREATMENT-RESEARCH-L@cornell.edu] On Behalf Of
Johnson, Will@DSS
Sent: Monday, June 17, 2002 7:33 PM
To: Child Maltreatment Researchers
Cc: Scott, Robert@DSS
Subject: RE: Pediatricians Alerted to the Dangers ...--Also, the
scientific prospects for safety assessment in general.

Aron,

You bring up a vitally important point which is crucial not just for
understanding the above example, but for understanding the scientific
prospects for the whole field of safety assessment, of which the above
example is a subset.  

When the rate of occurrence of an event is very low (like the death of a
child) it is virtually impossible to identify variables (like the
presence
of unrelated adults or any other variables) that will be found to have a
statistically significant relationship to the outcome of interest (i.e.
the
death of a child).  This is because of the low rate of occurence of the
outcome of interest (death).  The practical implication of this is that
no
child welfare worker (or researcher)can know what characteristics of a
case
fortell the later occurrence of a low base rate event like the death of
a
child.  Mathematically, in these situations, the outcome variable (e.g.
life
or death) is a constant (rather than a variable) function of the
independent
variable (e.g. the presence/absence of unrelated adults in the
household).
This means that the probability of occurrence of the outcome of interest
changes very little or not at all as a function of changes in the
independent variable.  This is because the dependent variable doesn't
vary
much at all--e.g.on a proportional basis, kids almost never die. So the
chances of a child dying are about the same--almost never does it
happen--whether there is an unrelated adult male in the home or not.

There is no known solution to this problem.  The work of Fuller and
Wells is
an example of how the problem of low base rates impedes the
identification
of variables that are useful in classifying cases as to the likelihood
of
occurrence of a low base rate event.  Using thousands of cases in an
analysis of maltreatment recurrence within 60 days of an initial
maltreatment incident (which has a very low base rate (3-5%)), Fuller
and
Wells (Children and Youth Services Review, Vol. 23, No. 1) did not
report a
single statistically significant relationship between any items on the
Illinois safety assessment (the CERAP) and maltreatment recurrence at 60
days. This was after Fuller and Wells altered their sample, consistent
with
their(quite reasonable)research design, to the point where the base rate
of
maltreatment recurrence within 60 days in their research sample
approached
50%.  This is far, far higher than the base rate in the stream of cases
flowing into CPS agencies, with which practitioners must work, and for
which
they would like to have a valid safety assessment--one that will
actually
tell them which kids are safe in the next 30 days or so, and which kids
aren't.

If (un)safety, or danger, is defined as imminent danger of harm and is
operationalized as serious injury or death within a short period of time
after an initial maltreatment incident, say 30 days, (30 days, rather
than
60, is probably more what people have in mind when they talk about a
time
horizon for "imminent danger") the base rate will probably be found to
be
substantially lower than 3%to 5%, perhaps below 1%, in the stream of
cases
coming into CPS.  Because of the base rate problem, there will never be
any
way to construct valid safety assessments useful in classifying cases as
to
the likelihood of serious injury or death with 30 (or 60) days.

Thus the base rate problem you mention, Aron, will prevent the
development
of what some in child welfare think should and should be "the next new
thing"--the development of predictively valid safety assessments. In
reality, any movement away from validated risk assessments in favor of
safety assessments that will be impossible to validate will leave child
welfare without any valid measurements of likelihood of future
maltreatment.

Will Johnson, Research Director,
California Child Welfare Structured Decision making Project.
-----Original Message-----
From: Aron Shlonsky [mailto:shlonsky@uclink4.berkeley.edu]
Sent: Monday, June 17, 2002 11:03 AM
To: Child Maltreatment Researchers
Subject: Re: Pediatricians Alerted to the Dangers to Children Living
With Men Unrelated to Them 


A strong word of caution here.  Death is a very rare event while poverty
is 
not, and the rate of "exposure" to households with unrelated adults is 
extremely high.  Most of us struggle to make ends meet even with a dual 
income.  I can't help but think that the rate of "false positives" 
(mistakenly labeling a family as high risk when they are not)  would be 
extremely high using just this crude measure (simply the presence of a 
person and nothing about the quality of that person). The unintended 
consequences of focusing policy and CPS response on just this item may
be 
unacceptable.  I'm not saying that's what is happening here or in these 
studies, but I cringe when I think about what could be...

Aron

At 09:04 PM 6/14/2002 -0700, you wrote:
>There has been a flurry in the news in recent weeks about a monograph
in
the
>April issue of the journal Pediatrics in which pediatricians are
alerted to
>the presence of a mother's boyfriend or stepfather as a risk factor for
>abuse of children. The article says that
>
>  Objective. Approximately 2000 children die annually in the United
States
>from maltreatment. Although maternal and child risk factors for child
abuse
>have been identified, the role of household composition has not been
>well-established. Our objective was to evaluate household composition
as a
>risk factor for fatal child maltreatment.
>
>Methodology. Population-based, case-control study using data from the
>Missouri Child Fatality Review Panel system, 1992-1994. Households were
>categorized based on adult residents' relationship to the deceased
child.
>Cases were all maltreatment injury deaths among children <5 years old.
>Controls were randomly selected from natural-cause deaths during the
same
>period and frequency-matched to cases on age. The main outcome measure
was
>maltreatment death.
>
>Results. Children residing in households with adults unrelated to them
were
>8 times more likely to die of maltreatment than children in households
with
>2 biological parents (adjusted odds ratio [aOR]: 8.8; 95% confidence
>interval [CI]: 3.6-21.5). Risk of maltreatment death also was elevated
for
>children residing with step, foster, or adoptive parents (aOR: 4.7; 95%
CI:
>1.6-12.0), and in households with other adult relatives present (aOR:
2.2;
>95% CI: 1.1-4.5). Risk of maltreatment death was not increased for
children
>living with only 1 biological parent (aOR: 1.1; 95% CI: 0.8-2.0).
>
>Conclusions. Children living in households with 1 or more male adults
that
>are not related to them are at increased risk for maltreatment injury
death.
>This risk is not elevated for children living with a single parent, as
long
>as no other adults live in the home. Pediatrics 2002; 109:615-621;
>epidemiology, injury, child abuse and neglect, fatalities, mortality,
>maltreatment, case-control study, death, risk factors, behavior.
>
>
>
>***********************************************************************
****
>  Niki Delson at Delson-Kokish Associates, P.O. Box 476, Trinidad, CA
95570
>         Clinical and Forensic Evaluations, Consultation & Training
>                  (707)677-3181-voice   (707)677-0187-fax
>                    niki@delko.net         www.delko.net
>***********************************************************************
****

Aron Shlonsky, MSW, MPH
Research Associate
UC Berkeley
School of Social Welfare
Center for Social Services Research
16 Haviland Hall #7400
Berkeley, CA  94720-7400
(510) 642-8139
(510) 642-1895 fax
shlonsky@uclink4.berkeley.edu