Skip to main content



Child-Maltreatment-Research-L (CMRL) List Serve

Browse or Search All Past CMRL Messages

Welcome to the database of past Child-Maltreatment-Research-L (CMRL) list serve messages (10,000+). The table below contains all past CMRL messages (text only, no attachments) from Nov. 20, 1996 - September 14, 2018 and is updated quarterly.

Instructions: Postings are listed for browsing with the newest messages first. Click on the linked ID number to see a message. You can search the author, subject, message ID, and message content fields by entering your criteria into this search box:

Message ID: 9026
Date: 2011-11-21

Author:Chaffin, Mark J. (HSC)

Subject:RE: Measuring racism

Jane Marie and list members, I want to revisit Jane Marie's original question, namely how do we MEASURE (emphasis added) racism in child welfare systems. From a measurement perspective, bias is less often directly measured and more often inferred or estimated from something analogous to a quasi-experiment. If you look across studies, the main way of approaching this is by comparing case decision making across cases. The ideal and conclusive comparison would be this one: Correct decision (y/n) for Case A (observed minority race condition) vs. Correct decision (y/n) for Case A (non-minority race counterfactual condition). Notice that both are Case A. A n/y contrast between the observed and counterfactual conditions supports an inference of racial bias--that is, the wrong decision is made under the minority condition, but the right one is made under the non-minority counterfactual. A y/y or n/n contrast would not support bias; although the n/n contrast would be something even more concerning than bias--systematic failure for everyone. A y/n contrast would suggest reverse bias. The estimation challenge, of course, is twofold--one cannot observe outcomes under both conditions (i.e. one case cannot be both observed and counterfactual, both minority and non-minority, at the same time), and one must have some way of knowing what the correct decision should have been for the case. There are two quasi-experimental methods issues here, neither of which IMO has been handled with much rigor in the majority of the disparity research, for some understandable reasons. The first is that not nearly enough data has been available on the cases to create reasonable counterfactuals. Studies may use only a few crude covariates like neighborhood or single parenting or poverty status that don't really give us a clear picture of the individual case risks, problems and resources. For example, studies may make the tenuous assumption that individuals in two given neighborhoods with similar levels of poverty have similarly distributed sets of problems, risks and resources and therefore should have similar patterns of dispositions. Does anyone who has worked in neighborhoods actually believe this? The related issue is that studies to my knowledge have not considered what the correct decision should have been in a particular case. They simply look for disparity on some action, such as foster care placement, irrespective of what "should" have happened under accepted best practices. This leaves open the debate we commonly hear about whether higher African American foster placement means that child welfare is unfair to African American parents, or whether child welfare is appropriately protecting African American children. What we want to know is whether those placements were the correct decision vs. whether those children would have been better served within their families. Until we get better case-by-case data to judge what best practice should have dictated for a particular case profile, the question is difficult to answer. There is a statistical issue concerning the use of regression covariance methods in these types of studies. Covariance does not estimate counterfactuals or create apples-to-apples comparisons--it "adjusts" estimates for the effects of a potential confounder under a set of assumptions which may not be realistic for our question here, and is often limited to estimating effects only at the mean of the covariate. Its not always a strong method. As another poster noted, the impact of poverty on actual maltreatment behavior is not equivalent across ethnic groups, and so introducing poverty as a regression model covariate may potentially exaggerate bias estimates. What we want are homogeneous strata or even matched pairs with respect to the clinical and case type variables that we believe are important for making "correct" decisions--safety, options and resources, including group-specific consideration for some risk factors such as poverty, and then make comparisons within those homogeneous strata defined by best practices. For example, we want strata with similar placement need attributable to parental substance use or depression, availability of extended family or social network supports, seriousness of parent-to-child violence, parental capacity to care for the child, family violence, etc. Creating homogeneous strata to examine the issue of bias is complicated and does involve some thorny decisions. For example, if no good family preservation services exist in a minority community, do you balance strata on that variable, treat that variable as one element of structural bias and therefore part of the independent variable, or use it as a covariate? If poverty is a big risk factor for serious parent-to-child violence in one group, but not the other, do you weight poverty the same or differently across groups? Lastly, there may be a base-rate complexity to this question. For example, if true maltreatment base rates are high in group A, and lower in group B, then a really sharp worker actually should evaluate objective risk indicators differently across the two groups (i.e. the sensitivity and specificity of the risk indicators will not be equal between groups). This might need to be factored into determining what the "correct" decision in a case should have been, because it may or may not be equivalent across groups. For example, there is a higher prevalence of sex abuse perpetration among men than among women. Therefore, we might evaluate exactly the same allegation against a mother as less probable than we might if it was against a father. One could (and many of the father advocates in the world would) call this bias or unfairness. Others might just call it objective Bayesian reasoning. To my knowledge, this issue has never been incorporated into bias measurement or disparity research, including in the vignette studies. All of this means that measuring racial bias in child welfare decision making is a complex issue--in some ways more analogous to messy quasi-experiments than to psychometrics or sociometrics. The reality of racial bias, historically or currently, is not what is in question here, and by pointing out these estimation issues my intent is not to encourage either racism denial or racism exaggeration. Rather, the issue is the accuracy and precision of our estimates, and I think we have a ways to go to measure this precisely. Mark Chaffin ________________________________________ From: NCCPR@aol.com [NCCPR@aol.com] Sent: Friday, November 18, 2011 4:42 PM Subject: Re: Measuring racism I would indeed disagree, based on the studies cited in my previous post and others, and also for reasons discussed in this Blog for the trade journal Youth Today: http://bit.ly/eDxHz6 Richard Wexler Executive Director National Coalition for Child Protection Reform 53 Skyhill Road (Suite 202) Alexandria VA 22314 703-212-2006 www.nccpr.org In a message dated 11/18/2011 4:51:57 P.M. Eastern Standard Time, tmac5528@yahoo.com writes: I believe the Drake articles reveal that there is no racism in the child welare system. The authors on the Pediatrics article are well respected. Anne's reference to the Bartholet reading suggests no racism as well. These are important findings. Maybe Dr. Wexler would disagree. TM --- On Fri, 11/18/11, Anne Day wrote: From: Anne Day Subject: Re: Measuring racism To: "Child Maltreatment Researchers" Cc: jane.marie.marshall@gmail.com Date: Friday, November 18, 2011, 1:56 PM I would also recommend looking at the literature surrounding the Multiethnic Placement Act of 1994 and “The Racial Disproportionality Movement in Child Welfare: False Facts and Dangerous Directions.” by Elizabeht Bartholet (http://www.law.harvard.edu/faculty/bartholet/RD_AZ_Final.pdf). On Fri, Nov 18, 2011 at 1:42 PM, > wrote: I recommend starting with Prof. Dorothy Roberts’ book, Shattered Bonds: The Color of Child Welfare (Basic Civitas Books: 2002). My organization’s Issue Paper, Child Welfare and Race, has citations for multiple studies. It’s available online at http://bit.ly/ckKv7M Some of the most important work in the field is being done by the Center for the Study of Social Policy. See in particular their Michigan Race Equity Review, available here: http://bit.ly/hnJR3j And while not a study, I highly recommend NPR’s recent three-part series on the impact of child protective services on Native Americans in South Dakota, available here: http://n.pr/uHDskg Richard Wexler Executive Director National Coalition for Child Protection Reform 53 Skyhill Road (Suite 202) Alexandria VA 22314 Office: (703) 212-2006 Cell: (703) 380-4252 www.nccpr.org Richard Wexler Executive Director National Coalition for Child Protection Reform 53 Skyhill Road (Suite 202) Alexandria VA 22314 703-212-2006 www.nccpr.org In a message dated 11/18/2011 9:34:12 A.M. Eastern Standard Time, jane.marie.marshall@gmail.com writes: Hi all, Could anyone direct me toward research that has undertook measuring racism in the child welfare and/or juvenile justice system? Thank you, Jane Marie Marshall Doctoral Candidate, School of Social Work University of Illinois at Urbana-Champaign Field Coordinator, Illinois Permanency Enhancement Project -- Anne Day anneday@sp2.upenn.edu anne.day@temple.edu Please consider the environment before printing this e-mail.

Jane Marie and list members, I want to revisit Jane Marie's original question, namely how do we MEASURE (emphasis added) racism in child welfare systems. From a measurement perspective, bias is less often directly measured and more often inferred or estimated from something analogous to a quasi-experiment. If you look across studies, the main way of approaching this is by comparing case decision making across cases. The ideal and conclusive comparison would be this one: Correct decision (y/n) for Case A (observed minority race condition) vs. Correct decision (y/n) for Case A (non-minority race counterfactual condition). Notice that both are Case A. A n/y contrast between the observed and counterfactual conditions supports an inference of racial bias--that is, the wrong decision is made under the minority condition, but the right one is made under the non-minority counterfactual. A y/y or n/n contrast would not support bias; although the n/n contrast would be something even more concerning than bias--systematic failure for everyone. A y/n contrast would suggest reverse bias. The estimation challenge, of course, is twofold--one cannot observe outcomes under both conditions (i.e. one case cannot be both observed and counterfactual, both minority and non-minority, at the same time), and one must have some way of knowing what the correct decision should have been for the case. There are two quasi-experimental methods issues here, neither of which IMO has been handled with much rigor in the majority of the disparity research, for some understandable reasons. The first is that not nearly enough data has been available on the cases to create reasonable counterfactuals. Studies may use only a few crude covariates like neighborhood or single parenting or poverty status that don't really give us a clear picture of the individual case risks, problems and resources. For example, studies may make the tenuous assumption that individuals in two given neighborhoods with similar levels of poverty have similarly distributed sets of problems, risks and resources and therefore should have similar patterns of dispositions. Does anyone who has worked in neighborhoods actually believe this? The related issue is that studies to my knowledge have not considered what the correct decision should have been in a particular case. They simply look for disparity on some action, such as foster care placement, irrespective of what "should" have happened under accepted best practices. This leaves open the debate we commonly hear about whether higher African American foster placement means that child welfare is unfair to African American parents, or whether child welfare is appropriately protecting African American children. What we want to know is whether those placements were the correct decision vs. whether those children would have been better served within their families. Until we get better case-by-case data to judge what best practice should have dictated for a particular case profile, the question is difficult to answer. There is a statistical issue concerning the use of regression covariance methods in these types of studies. Covariance does not estimate counterfactuals or create apples-to-apples comparisons--it "adjusts" estimates for the effects of a potential confounder under a set of assumptions which may not be realistic for our question here, and is often limited to estimating effects only at the mean of the covariate. Its not always a strong method. As another poster noted, the impact of poverty on actual maltreatment behavior is not equivalent across ethnic groups, and so introducing poverty as a regression model covariate may potentially exaggerate bias estimates. What we want are homogeneous strata or even matched pairs with respect to the clinical and case type variables that we believe are important for making "correct" decisions--safety, options and resources, including group-specific consideration for some risk factors such as poverty, and then make comparisons within those homogeneous strata defined by best practices. For example, we want strata with similar placement need attributable to parental substance use or depression, availability of extended family or social network supports, seriousness of parent-to-child violence, parental capacity to care for the child, family violence, etc. Creating homogeneous strata to examine the issue of bias is complicated and does involve some thorny decisions. For example, if no good family preservation services exist in a minority community, do you balance strata on that variable, treat that variable as one element of structural bias and therefore part of the independent variable, or use it as a covariate? If poverty is a big risk factor for serious parent-to-child violence in one group, but not the other, do you weight poverty the same or differently across groups? Lastly, there may be a base-rate complexity to this question. For example, if true maltreatment base rates are high in group A, and lower in group B, then a really sharp worker actually should evaluate objective risk indicators differently across the two groups (i.e. the sensitivity and specificity of the risk indicators will not be equal between groups). This might need to be factored into determining what the "correct" decision in a case should have been, because it may or may not be equivalent across groups. For example, there is a higher prevalence of sex abuse perpetration among men than among women. Therefore, we might evaluate exactly the same allegation against a mother as less probable than we might if it was against a father. One could (and many of the father advocates in the world would) call this bias or unfairness. Others might just call it objective Bayesian reasoning. To my knowledge, this issue has never been incorporated into bias measurement or disparity research, including in the vignette studies. All of this means that measuring racial bias in child welfare decision making is a complex issue--in some ways more analogous to messy quasi-experiments than to psychometrics or sociometrics. The reality of racial bias, historically or currently, is not what is in question here, and by pointing out these estimation issues my intent is not to encourage either racism denial or racism exaggeration. Rather, the issue is the accuracy and precision of our estimates, and I think we have a ways to go to measure this precisely. Mark Chaffin ________________________________________ From: NCCPRaol.com [NCCPRaol.com] Sent: Friday, November 18, 2011 4:42 PM Subject: Re: Measuring racism I would indeed disagree, based on the studies cited in my previous post and others, and also for reasons discussed in this Blog for the trade journal Youth Today: http://bit.ly/eDxHz6 Richard Wexler Executive Director National Coalition for Child Protection Reform 53 Skyhill Road (Suite 202) Alexandria VA 22314 703-212-2006 www.nccpr.org In a message dated 11/18/2011 4:51:57 P.M. Eastern Standard Time, tmac5528yahoo.com writes: I believe the Drake articles reveal that there is no racism in the child welare system. The authors on the Pediatrics article are well respected. Anne's reference to the Bartholet reading suggests no racism as well. These are important findings. Maybe Dr. Wexler would disagree. TM --- On Fri, 11/18/11, Anne Day wrote: From: Anne Day Subject: Re: Measuring racism To: "Child Maltreatment Researchers" Cc: jane.marie.marshallgmail.com Date: Friday, November 18, 2011, 1:56 PM I would also recommend looking at the literature surrounding the Multiethnic Placement Act of 1994 and “The Racial Disproportionality Movement in Child Welfare: False Facts and Dangerous Directions.” by Elizabeht Bartholet (http://www.law.harvard.edu/faculty/bartholet/RD_AZ_Final.pdf). On Fri, Nov 18, 2011 at 1:42 PM, > wrote: I recommend starting with Prof. Dorothy Roberts’ book, Shattered Bonds: The Color of Child Welfare (Basic Civitas Books: 2002). My organization’s Issue Paper, Child Welfare and Race, has citations for multiple studies. It’s available online at http://bit.ly/ckKv7M Some of the most important work in the field is being done by the Center for the Study of Social Policy. See in particular their Michigan Race Equity Review, available here: http://bit.ly/hnJR3j And while not a study, I highly recommend NPR’s recent three-part series on the impact of child protective services on Native Americans in South Dakota, available here: http://n.pr/uHDskg Richard Wexler Executive Director National Coalition for Child Protection Reform 53 Skyhill Road (Suite 202) Alexandria VA 22314 Office: (703) 212-2006 Cell: (703) 380-4252 www.nccpr.org Richard Wexler Executive Director National Coalition for Child Protection Reform 53 Skyhill Road (Suite 202) Alexandria VA 22314 703-212-2006 www.nccpr.org In a message dated 11/18/2011 9:34:12 A.M. Eastern Standard Time, jane.marie.marshallgmail.com writes: Hi all, Could anyone direct me toward research that has undertook measuring racism in the child welfare and/or juvenile justice system? Thank you, Jane Marie Marshall Doctoral Candidate, School of Social Work University of Illinois at Urbana-Champaign Field Coordinator, Illinois Permanency Enhancement Project -- Anne Day annedaysp2.upenn.edu anne.daytemple.edu Please consider the environment before printing this e-mail.