Can a Test Really Tell Who’s Dangerous?

A psychological test that has influenced sentencing and parole decisions for decades is unreliable and out of date, research from U of T Mississauga suggests, raising questions about its continued use in the justice system.

Rasmus Rosenberg Larsen, an assistant professor of forensic epistemology and philosophy of science, is a critic of the diagnosis commonly known as psychopathy, or Psychopathic Personality Disorder.

In a study that analyzed a test commonly used to evaluate psychopathy, Larsen and several co-authors found it was often being improperly applied and suggested courts exercise caution when dealing with cases involving the tool. “A psychopathy assessment can influence very high-stakes decisions,” says Larsen, who specializes in psychology and law. “The highest stakes would be capital sentencing in the United States, but there are also things like parole. Does a person get to go home to see their family on a regular basis or not?”

In a study published in Psychology, Public Policy, and Law, Larsen and several co-authors examined 3,315 Canadian cases in which any type of psychopathy assessment was used. Nearly 95 per cent of those assessments relied on the Hare Psychopathy Checklist-Revised, or PCL-R – most often introduced by prosecutors. The test is used to make a clinical inference such as “diagnosing” a defendant with psychopathy.

The study, supported by the Connaught Research Foundation, found the use of psychopathy assessments grew more than ninefold between 2000 and 2013. That was followed by a decline of 63 per cent over the next decade.

Larsen says the decline could reflect growing concerns over the test, which uses a checklist with scores from 0 to 40. Scores above 25 are considered to indicate clinically significant levels of psychopathy. “The test was used like this magical tool where if you scored high, you could put that person in the bucket: the worst of the worst,” says Larsen. “It implies there are so many things wrong with the accused – they are at high risk of reoffending and can’t be treated.”

Larsen says one problem is that the manual accompanying the checklist, first published in 1980, has not been updated since 2003. Since the last revision, he notes, field research has found limited predictive value in the test’s scores, especially when used with U.S.-based, male or minority populations. As such, many researchers – including test creator, Canadian forensic psychologist Robert Hare – now caution against using it in court.

Hare warned that to be reliable the test must be conducted by a professional with significant training and clinical experience, be based on high quality information and include ratings from two independent clinicians.

As early as 1998, he stressed that it’s difficult to meet all those conditions in a court setting, sometimes because of “adversarial allegiance” – where clinicians are giving different scores depending on whether they were hired by the prosecution or the defence. “An expert witness is supposed to be independent, not swayed by who retained them,” says Larsen. “However, this is exactly what we see.”

Larsen adds he’s encountered swift assessments based only on records and not interviews.

The manual also describes psychopathy as an untreatable condition, a position generally dismissed by contemporary researchers after more recent studies showed that rehabilitation programs can have positive effects.

Another concern, says Larsen, is that the assessments may prejudice judges and jurors against the accused. “When you call it a psychopathy test, you’re triggering all sorts of biases because many people intuitively assume that when you say ‘psychopath,’ we’re talking about the absolute worst,” says Larsen. “The evidence does not suggest that, but those are nevertheless the semantic connotations of a label like that.”

Larsen argues that the test’s risks now outweigh its benefits. Since publishing his research, he says he’s been contacted by attorneys wanting to understand the relevance of the findings to current cases and has seen an increasing willingness to scrutinize and challenge the test. “However, I do not want to overstate this shift,” he says. “The growing skepticism is real, but our study also found that psychopathy evidence – and the PCL-R in particular – was introduced overwhelmingly by the Crown and formally challenged in only about five per cent of the cases we examined.”

The research findings do not automatically create a clear pathway for appeal or rehearing of a case, Larsen says. This would depend on the procedural history of the case, the consequences for the individual, and the extent to which the checklist influenced the decision. “In some cases, our research may provide defence counsel with grounds to initiate some form of appeal process,” he explains, “but that would need to be established case by case.”

More broadly, however, Larsen says he’d like to see courts ask tougher questions before admitting or relying on expert evidence involving the test. “A Canadian defence attorney who is familiar with our study and well versed in the scientific study of psychopathy should have a strong basis for persuading a judge that this evidence ought not to be admitted,” says Larsen. “In short, the court does not need this test.”