Illinois drops predictive analytics for at-risk children
Connecting state and local government leaders
The predictive analytics program to identify children at risk for serious injury or death "didn't seem to be predicting much," the agency's director said.
The Illinois Department of Children and Family Services has decided to end a program using predictive analytics to identify children at risk for serious injury or death because it "didn't seem to be predicting much," DCFS Director Beverly Walker told the Chicago Tribune.
The service -- administered by nonprofit Eckerd Connects and its for-profit partner, Mindshare Technology – assigned a risk score to children who were the subject of an abuse allegation to the agency's hotline and then rated the children's risk of harm in the next two years.
But rather than help evaluate risk to children, the system overwhelmed caseworkers, flagging 4,100 children as facing a 90 percent or greater probability of death or injury. And it did not predict deaths of children who had been subject of several abuse investigations, according to the Tribune.
The missteps in Illinois were the result of errors on the part of both Eckerd and DCFS. A combination of data entry errors, missing information and regulations that forced the agency to erase "unfounded" mistreatment claims impaired the reliability of the data the system worked with.
Additionally, the alerts sent by the system were confusing and should have been sent to DCFS supervisors, not front-line caseworkers who were not trained on the Eckerd system.
A spokesperson for Eckerd told the paper it is changing its language for the alerts so they're focused less on predicting what will happen in the future and instead on how the risk correlates to what’s happened in historical cases. It also argued that the number of children marked as high risk might seem high, but it was a small percentage of the overall cases analyzed.
These types of predictive systems have been deployed by agencies across the country. The Department of Social Services in New Hanover County, N.C., started using predictive analytics software in 2016 to identify children most at risk of abuse and prioritize social workers’ caseloads. The Oklahoma Department of Human Services also began using Eckerd services last year. In 2014, the Florida’s Department of Children and Families began investigating predictive analytics.
Doug Tobin, an Eckerd Connects spokesperson, responded to the Tribune story on the company’s website. The company stands by its algorithm.
“The Illinois Department of Children and Family Services acknowledged the data they provided to Eckerd Connects had multiple errors,” Tobin wrote. “We also believe if Eckerd Connects had received more accurate information the results would have been different.”
NEXT STORY: Opioid code-a-thon winners announced