As Child Welfare Agencies Turn to Data Analytics ‘We Have to Be Really Careful’

Callers phoned in about 17,000 complaints of suspected child abuse and neglect last year to an Allegheny County Department of Human Services hotline, according to the agency’s director.

That averages out to about 46 calls per day. Located in southwestern Pennsylvania, Allegheny County has roughly 1.2 million residents and includes Pittsburgh and many of its neighboring communities. Answering those hotline calls in the county is a team of about 12 to 15 screeners working in shifts. They have to recommend to their supervisors whether the department should investigate each alleged report of mistreatment.

“It’s just a tremendous amount of volume, so you can imagine how quickly those decisions have to be made,” said Erin Dalton, deputy director for the Department of Human Service’s Office of Data Analysis, Research and Evaluation.

And the stakes are high. A wrong decision about whether to investigate a complaint could mean a child in danger goes overlooked. Or it could trigger an inquiry into a family where there’s been no wrongdoing, which can trouble kids and waste department staff time.

Call screeners base their recommendations on the seriousness of an allegation as well as other information.

For instance, they can check to see if a family has been referred to the department over a child welfare issue in the past, if a parent has received publicly funded mental health treatment, or been booked in jail. After examining these kinds of factors, a screener comes up with a rating of risk and safety to accompany their recommendation.

But, since August, they’ve taken an extra step using a predictive data analytics tool.

The Allegheny Family Screening Tool pulls information from the department's "data warehouse," a sprawling computer database system that contains more than one billion records related to over one million people. Accessible through the data warehouse is information from 29 sources. In addition to the county human services department, these sources include public entities like housing authorities, school districts and the criminal justice system.

Weighing over 100 variables, the analytics tool comes up with a “family screening score” for households at the center of an abuse or neglect complaint. Data for each person involved in a complaint is used to generate the overall score. A higher score implies a greater chance of a future "event," such as abuse, or placement in foster care.

The variables the tool uses to produce the family screening score are not unlike factors the screeners consider. In this case, however, it's the computer algorithm gauging risk. Screeners include the score along with the other information they've traditionally passed along to their supervisors about a call.

“You don’t screen out something that was really high risk, but based on the call you wouldn’t have known it,” said Marc Cherna, director of the county's Department of Human Services.

“If we can do something that's going to improve our decision making, that's worth it,” he added. “Especially on this front-end piece, because it can save children’s lives.”

‘We Have to be Really Careful’

State and local governments are increasingly turning to data analytics to guide decision making in areas ranging from restaurant inspections to criminal justice. But using algorithms to assess child welfare is a newer frontier—one that is especially sensitive.

“This magnifies all of the bias that’s already built into child welfare and is extremely dangerous,” Richard Wexler, executive director of the National Coalition for Child Protection Reform, an Alexandria, Virginia-based nonprofit, said of advanced data analytics.

Wexler highlighted court system algorithms used to assign risk scores to criminal defendants and how some of these models have come under scrutiny for racial bias.

Applying predictive analytics to child protective services, he believes, could produce similar results, unfairly targeting poor and minority families. Another potential pitfall Wexler sees is that analytics tools might place unwieldy burdens on agencies.

“They’ll actually wind up missing more kids in danger because they’ll be so overloaded with the false positives,” he said.

The contract for the predictive analytics program in Allegheny County was awarded through a competitive process to a team of academics led by professor Rhema Vaithianathan, of New Zealand’s Auckland University of Technology.

Emily Putnam-Hornstein, a professor at the University of Southern California’s Suzanne Dworak-Peck School of Social Work, is among the researchers involved in the project.

“We have to be really careful,” she said as she discussed analytics models for child welfare agencies.

But in her view: “We can either keep doing the same thing that we’ve been doing; and I don’t think many people would say that the child protection systems here in the U.S. are working perfectly. Or we can explore whether, on the margin, data and analytics can help us move closer to a system where we think that the kids who are at greatest risk are correctly being identified.”

Cherna stressed that Allegheny County’s use of predictive analytics for child protective services is confined to the “go” or “no-go” decisions about whether to investigate a report of abuse or neglect. Screeners lock in the risk and safety rating they come up with for a family before running the computer model. And investigators don’t see the risk scores the algorithm produces.

“We don’t want to bias the investigative worker,” Cherna said.

If on a 1-to-20 scale of risk, a score is 20, he said, “they’ll be more likely to remove those kids because they won’t want to take a chance.”

‘All the Help We Can Get’

Wedged between the Cape Fear River and the Atlantic Ocean, New Hanover County, North Carolina has about 220,000 residents.

The county is on track to be one of the first in North Carolina to use advanced analytics to monitor risk factors for kids. It plans to do so initially as part of a pilot program, with a tool developed by Cary, North Carolina-based SAS Analytics. The pilot is slated to be up and running in 12 to 18 months.

According to Wanda Marino, the county’s assistant director of social work services, $800,000 of funding for the analytics pilot program is coming from the The Duke Endowment, headquartered in Charlotte. SAS, she said, is investing roughly $1 million toward the effort. And if the county adopts the program permanently, it’s expected to cost around $100,000 annually.

“With numbers growing as far as child welfare, abuse, neglect cases that we’re handling,” Marino said, “we need all the help we can get and we need advanced tools.”

The program in New Hanover is envisioned differently than the one in Allegheny County.

As described by Marino and Brian Bocnuk, child welfare program manager at the county’s department of social services, some of its key features will be designed to help caseworkers prioritize their clients.

“A social worker will come in, open up their desktop, and pull up their whole caseload, and the cases will be arranged according to risk,” Bocnuk said.

Marino gave another example of how the system could function.

A family might be working with the department on a treatment and safety plan after a domestic violence incident, she said. The father was the aggressor and can’t be left alone with his kids. The household’s situation is considered medium risk by the department.

But then one night there’s a repeat episode, the father gets out of control and the police are called. The way it is now, the department of social services would rely on the police to tell them about the incident.

“Sometimes that happens, sometimes it doesn’t,” Marino said.

With the analytics system in place, she explained, the department would get an alert that risk for the household has increased. “And,” she added, “we’re going to be responding appropriately.”

‘Community Is Entitled to Know’

Allegheny County and New Hanover County are not the only jurisdictions moving forward with child protective services data analytics programs.

SAS is working directly with three other organizations using advanced analytics to assess the risk of child fatalities and mistreatment, company spokesman Trent Smith said in an email last week. He said one of these projects is in Florida, but declined to offer more details.

Meanwhile, Eckerd Kids, one of the nation’s largest nonprofit child and family services organizations, has also developed an analytics platform, working with a company called MindShare Technology. This effort began in Hillsborough County, Florida in 2013.

In June, the Oklahoma Department of Human Services said it would pilot Eckerd’s Rapid Safety Feedback analytics tool. And, according to an Eckerd fact sheet, the organization has also been active developing similar systems in Alaska, Connecticut and Maine.

“What’s definitely needed in all cases is transparency,” said Jay Stanley, a senior policy analyst with the American Civil Liberties Union, as he discussed the use of child welfare data analytics systems. “There’s no long history of deployments and expertise.”

Putnam-Hornstein offered a similar perspective.

“I’ve definitely been kind of beating the table about the transparency thing,” she said.

While explaining the statistics behind predictive algorithms to non-experts might be unrealistic, Putnam-Hornstein thinks vendors of data analytics tools should allow independent analysis of how accurately their models perform.

“The community is entitled to know,” she said.

Allegheny County’s Dalton agrees. Her department has “full visibility” into the mechanics of its algorithm. “I think that’s important,” she said. “A lot of these companies are not transparent about the model.” Dalton added: “You don’t know what’s in the box.”

In addition to transparency, paying attention to how data analytics models mesh with child welfare workers is important, according to Putnam-Hornstein.

“We can feed risk scores to people all over the place,” she said. “But if they don’t believe the risk score, if they don’t properly use the risk score, if we haven’t thought about who should have access to the risk score, then there can be a lot of, I think, unintended consequences."