It turns out that a trusted crime-fighting algorithm used to predict if criminals will re-offend might not be any better at its job than a random untrained human.

The technology has already been widely used in the U.S. by judges as a risk assessment tool when deciding on an offender’s sentence. Predictive technology used to stem crime has since its inception provoked fear and suspicion, whether that is police using big data in “Minority Report” fashion to prevent crime before it happens or to understand a criminal’s future intentions.

In the case of the latter, something called the COMPAS algorithm has been used by the courts for some time to detect the possibility of recidivism after people have been charged with a crime. The technology was welcomed with open arms within the justice system and soon judges were consulting COMPAS before sentencing an offender. Investigations, however, have revealed that it and other such automated decision-making systems are not flawless.

Far from it, say two researchers at Dartmouth College, Julia Dressel and Hany Farid, whose recent article in the journal Science Advances revealed that the algorithm is no better than any average human at spotting recidivism.

“There was essentially no difference between people responding to an online survey for a buck and this commercial software being used in the courts,” said Farid. “If this software is only as accurate as untrained people responding to an online survey, I think the courts should consider that when trying to decide how much weight to put on them in making decisions.”

Dressel said there’s little proof that algorithms are any more successful at making predictions as humans, adding that the nascent industry of crime predicting is secretive when it comes to sharing its results. That’s why she and Farid devised their own questionnaire to give to the public via Amazon.com Inc.’s Mechanical Turk online marketplace.

The main difference is that the COMPAS algorithm uses 137 data points to assess criminals, but the questionnaire the researchers sent out to the public only contained seven. The 1,000 defendants used were all real, taken from Broward County in Florida. When the answers came back, respondents showed a 67 percent accuracy rate on average, which was 2 percent better than COMPAS.

“How can it be that this software that is commercially available and being used broadly across the country has the same accuracy as Mechanical Turk users?” said Farid.

Equivant, the company behind the COMPAS algorithm, had issues with the study but also said the research actually validates the technology. “Instead of being a criticism of the COMPAS assessment, [it] actually adds to a growing number of independent studies that have confirmed that COMPAS achieves good predictability and matches,” the company said in a statement.

The Dartmouth researchers are not so sure, saying that when the sentencing of an offender is involved, predictability at the level of untrained humans on an online marketplace might not in fact legitimize the algorithm’s use in the justice system. Moreover, defendants don’t always know they are being assessed by such technologies, which have been decried as being the justice system’s “secretive algorithms.”

Last year a defendant in the U.S. challenged the use of COMPAS after finding out his sentence was longer because of his COMPAS score. He said this was in violation of his constitutional rights and COMPAS was not a valid technology for risk assessment. The court declined to hear the case, even though the defendant argued that the formula for the risk assessment had not been revealed to him. In the U.S. judges are usually required to explain their decisions.

Image: Jose via Flickr