By Sally Ho and Garance Burke
April 29, 2022 GMT
https://apnews.com/article/child-welfare-algorithm-investigation-9497ee937e0053ad4144a86c68241ef1
Inside a cave fortress in downtown Pittsburgh, attorney Robin Frank defends parents at one of their lowest points – when they risk losing their children.
The job is never easy, but in the past she knew what she faced when she opposed child protection services in the family court. Now she’s worried she’s struggling with something she can’t see: an opaque algorithm whose statistics help social workers decide which families should be investigated in the first place.
“A lot of people don’t even know it’s being used,” Frank said. “Families should have the right to have all the information in their file.”
From Los Angeles to Colorado and across Oregon, as child protection agencies use or consider tools similar to Allegheny County, Pennsylvania, an Associated Press review has identified a number of concerns about the technology, including its reliability and potential. strengthening racial differences in the child protection system. Related issues have already disrupted plans in some jurisdictions to use predictive models, such as the Illinois tool.
According to a new study by a team from Carnegie Mellon University, obtained exclusively from the AP, Allegheny’s algorithm in its first years of operation showed a model for marking a disproportionate number of black children for “mandatory” neglect investigation compared to white children. Independent researchers who received data from the county also found that social workers disagreed with the risk estimates that the algorithm produces about a third of the time.
County officials said social workers could always cancel the tool and called the study “hypothetical.”
Allegheny County Child Protection, the cradle of Mr Rodgers’ television district and the iconic innovations aimed at children say the state-of-the-art tool – which is attracting nationwide attention – is using data to support agency staff. while trying to protect children from neglect. This nuanced term can include everything from inadequate housing to poor hygiene, but it is a different category from physical or sexual violence, which is investigated separately in Pennsylvania and is not subject to the algorithm.
“Workers, whoever they are, should not be invited to make 14, 15, 16,000 of these kinds of decisions with incredibly imperfect information in a given year,” said Erin Dalton, director of the county’s human services department and a pioneer in the application of the algorithm for predicting the welfare of children.
____
This story, backed by the Pulitzer Center for Crisis, is part of the Associated Press’s ongoing series, Tracked, which explores the power and effects of algorithm-driven solutions on people’s daily lives.
____
Critics say he gives a program based on data collected mainly on poor people a huge role in deciding the fate of families, and they warn against the growing reliance of local employees on artificial intelligence tools.
If the instrument had acted alone to screen at a comparable frequency of calls, it would have recommended that two-thirds of black children be investigated, compared to about half of all other children reported, according to another study published last month and co-authored by a researcher who audited the county’s algorithm.
Proponents worry that if similar tools are used in other child protection systems with minimal or no human intervention – similar to how algorithms have been used to make decisions in the criminal justice system – they could exacerbate existing racial differences. in the child welfare system.
“It doesn’t reduce the impact on black families,” said Logan Stapleton, a researcher at Carnegie Mellon University. “In terms of accuracy and inconsistency (the district) makes strong statements that I think are misleading.
Because family court hearings were closed to the public and records were sealed, the AP failed to identify first-hand families that the algorithm recommended must be investigated for child neglect, or cases that led to a child being sent to a foster family. care. Families and their lawyers can never be sure of the role of the algorithm in their lives because they are not allowed to know the results.
SAFER, FASTER
Potential neglect incidents have been reported to the Allegheny County Child Protection Hotline. The reports go through a screening process in which the algorithm calculates the potential risk to the child and assigns an estimate. Social workers then use their judgment to decide whether to investigate.
The Allegheny family screening tool is specifically designed to predict the risk of a child being placed in foster care within two years of being studied. Using a wealth of detailed personal data collected from birth, Medicaid, drug abuse, mental health, prisons and probation records, among other government datasets, the algorithm calculates a risk assessment of 1 to 20: The higher the number, the more the greater the risk.
Given the high stakes – missing a neglect report can result in the death of a child, but careful study of family life can lead to separation – the county and the developers suggest that their tool can help “the right course.” “And make the agency’s work more in-depth and effective by eliminating useless reports so that social workers can focus on children who really need protection.
The developers describe the use of such tools as a moral imperative, saying that child protection officers should use everything they have to make sure that children are not neglected.
“There are children in our communities who need protection,” said Emily Putnam-Hornstein, a professor at the University of North Carolina at Chapel Hill School of Social Work who helped develop the Allegheny tool by speaking at a virtual panel hosted by New York University in November.
Dalton said algorithms and other forecasting technologies also provide a scientific test of the personal preferences of call center employees, as they see the risk assessment when deciding whether a case deserves investigation. If the case escalates, Dalton said the full investigation is being conducted by a different social worker who personally checks, decides if the allegations are true and helps determine if the children should be placed in foster care.
CMU researchers found that from August 2016 to May 2018, the tool calculated results that suggest that 32.5% of black children reported to have been neglected should be subject to a “mandatory” investigation. , compared to 20.8% of white children.
In addition, the county confirmed to the AP that for more than two years, a technical problem with the tool has sometimes given social workers wrong results, or underestimating or overestimating the risk to a child. District officials said the problem has been fixed since then.
The county did not dispute the figures of CMU researchers, but Dalton said the research paper was “a hypothetical scenario that is so far removed from the way this tool has been implemented to support our workforce.”
The CMU study found no difference in the percentage of black families studied after the adoption of the algorithm. The study found that workers were able to reduce this discrepancy produced by the algorithm.
The county says social workers are always on the line and ultimately responsible for deciding which families to investigate because they can repeal the algorithm, even if it marks a case for mandatory investigation. Dalton said the tool would never be used alone in Allegheny, and he doubted a county would allow fully automated decisions about family life.
“Of course they could do that,” she said. “I think they are less likely to do that because there is no real point in doing so.”
Despite what the county describes as a precaution, a child welfare expert who worked for an Allegheni County contractor says there is still cause for concern.
“When you have man-made technology, the bias will be in the algorithms,” said Nicole Lee Beadle, who has worked for nearly a decade in child welfare, including as a family therapist and foster care specialist at Allegheny. county “If they’ve designed the perfect tool, it really doesn’t matter because it’s designed with a lot of imperfect data systems.”
Biddle is a former foster child who has become a therapist, social worker and political advocate. She left in 2020, largely due to her growing frustration with the child protection system. She also said officials had dismissed her concerns when she asked why the families were originally directed to investigate.
“We could see the report and this decision, but we never saw the real tool,” she said. “I will be greeted with …” What does this have to do with anything now? “
Movements for redesigning – or dismantling – child protection services have grown in recent years as generations of horrific foster care outcomes have been rooted in racism.
In a note last year, the US Department of Health and Human Services cited racial differences “in almost every major decision-making moment” in the child protection system, an issue that Aisha Schomburg, deputy commissioner of the US Bureau of Children, points out that it leads more than half of all black children across the country to be investigated by social workers. “Excessive surveillance leads to mass separation of the family,” Schomburg wrote in a recent blog post.
With discussions on race and equality circulating in child protection circles, Putnam-Hornstein attended a roundtable of experts convened by the conservative American Enterprise Institute last fall and co-authored a paper criticizing advocates who believe that child protection systems are …
Add Comment