February 17, 2011

Warren Horinek did not murder his wife. That’s what he said, that’s what the medical examiner said, that’s what the homicide sergeant said. Even the district attorney’s office in the Horineks’ hometown of Ft. Worth, Texas, agreed that he was innocent—not something a Texas prosecutor typically says. But when Bonnie Horinek died in 1995, her parents refused to believe what the evidence strongly suggested—that Bonnie shot herself—and instead they enlisted the services of a blood-spatter analyst to prove that it was their son-in-law who had killed their daughter.

The spatter analyst zeroed in on the blood-soaked T shirt Horinek was wearing when the paramedics arrived. To him, the fine spray of blood on Horinek’s left shoulder was not from administering CPR, as Warren said it was, and as the 911 recording seemed to indicate, but from shooting Bonnie at close range. On the basis of that testimony, Horinek was convicted of murder and sentenced to 30 years. But did they really get their man? Horinek’s lawyers have filed a writ of habeas corpus to try to have him released; much of the spatter analyst’s testimony, the lawyers argue, “was contrary to known and accepted science.”

In the age of CSI and Dexter, we’re led to believe that forensic science is a high-tech discipline, powerful and sophisticated enough to catch any criminal.

As it turns out, whether blood-spatter analysis and disciplines like it qualify as “science” at all is a matter of increasing debate. In a sharply critical report issued in 2009, the National Academy of Sciences said, “The simple reality is that the interpretation of forensic evidence is not always based on scientific studies.” Taking aim at disciplines as varied as ballistics, hair and fiber analysis, bite-mark comparison—even fingerprints—the report declared, “This is a serious problem.”

The last few years have seemed to bear out the report. Dozens of elite crime labs all over the country, from Nassau County, N.Y., to San Francisco, to Virginia, Cleveland, Oklahoma, and Baltimore, have been involved in scandals involving mishandled evidence and false or misleading forensic testimony. This past summer, a North Carolina attorney general’s audit discovered that the state’s Bureau of Investigation had withheld or distorted evidence in more than 200 cases.

Even some of the best funded and most sophisticated crime-fighting organizations are being taken to task for their use of forensic evidence. This week, the New York Times reported that the Federal Bureau of Investigation had “overstated the strenght of genetic analysis” during the investigation of Bruce E. Ivins, who allegedly mailed anthrax to newsrooms and Senate offices in the wake of the 9/11 attacks.

A year-long investigation by the independent journalism nonprofit ProPublica revealed major problems in the nation’s coroner system: pathologists not certified in pathology, physicians who flunk their board exams, even coroners who are not physicians at all. “In nearly 1,600 counties across the country,” the investigation found, “elected or appointed coroners who may have no qualifications beyond a high-school degree have the final say on whether fatalities are homicides, suicides, accidents or the result of natural or undetermined causes.”

For his forthcoming book, Convicting the Innocent: Where Criminal Prosecutions Go Wrong (Harvard University Press, April 2011), University of Virginia law professor Brandon Garrett examined the trial transcripts and other legal documents of the first 250 people to be exonerated by DNA in this country. He discovered that in more than half these cases, trials were tainted by “invalid, unreliable, concealed, or erroneous forensic evidence.” The errors ranged from analysts making up statistics on the fly, implying that their methods were more scientific than they actually were, and exaggerating or distorting their findings to support the prosecution.

Peter Neufeld, a lawyer in New York and cofounder of the Innocence Project, which has helped to facilitate many of these exonerations, calls it the “elastic expert: no matter what you see, I can distort it so that it would be a match.”

This “elasticity” is possible because the tests are largely subjective. Just how much human judgment is required depends on the discipline: DNA testing is mostly—though not entirely—done by machine, for instance, whereas microscopic hair comparison is based solely on the analyst’s opinion. Even fingerprints, which many of us regard as foolproof tools for identifying culprits—think Dexter feeding a print into his computer and a bad guy’s photo and driver’s license appearing on the screen—in fact rely largely on human interpretation, and therefore are subject to human error.

One of the most famous examples of the danger of fingerprints was the case of Oregon lawyer Brandon Mayfield, arrested in 2004 in the wake of the Madrid train bombings. Working from a partial print that Spanish authorities had found on a plastic bag of detonators, several top FBI analysts declared Mayfield’s print a match. That is, until Spanish authorities identified Ouhnane Daoud, now wanted for terrorism in connection to the crime. When it became clear that Daoud’s prints were a much better match, the FBI was forced to admit that its own bias and “circular reasoning” had led them to Mayfield, who had no involvement in the bombings.

Part of the problem is what social scientists call “context bias.” Most forensics labs are located within police departments, so analysts may see themselves as working “for” the prosecution. They also usually have information about the evidence they’re testing—for example, that the suspect has a prior record. “There’s a lot of research to suggest that knowledge could have biasing effect,” says Jennifer Mnookin, a professor at the UCLA School of Law.

In a recent Supreme Court case, Justice Antonin Scalia, writing for the majority, said that whether consciously or not, an analyst “responding to a request from a law enforcement official may feel pressure—or have an incentive—to alter the evidence in a manner favorable to the prosecution.” The judges’ ruling means that forensic test results may be subject to the same kind of scrutiny as any other evidence, and an analyst from the lab that ran the test must be present in court to be cross-examined, just like any other witness.

“Obviously, most people in this community are trying to do their jobs well and are not trying to frame innocent people,” says the University of Virginia’s Garrett. “But what we’ve seen come out of these exoneration cases and in additional scandals at the laboratories is that this is not a problem of a few bad apples. Who is the competent analyst that can testify about a technique that’s fundamentally unreliable? That’s not a bad-apple problem. That’s a serious problem with our entire system.”

At the heart of these criticisms is the issue of what scientists call validity and reliability. A test is valid if its results are factually accurate. A test is reliable if multiple tests will lead to the same conclusion. Some forensics tests, like blood typing, are very reliable: no matter how many times your doctor draws your blood, you will always have the same blood type. Occasionally there are mistakes, of course, but they are predictable: blood-typing tests have well-documented and well-understood error rates. Others, like hair comparison, are unreliable: studies have shown that multiple technicians examining the same two hairs—even the same technician examining the same two hairs at different times—come to multiple conclusions. Critics say that many of forensic science’s most basic tools are neither reliable nor valid.

For example, at the trial of Jimmy Ray Bromgard, who served more than 14 years of a 40-year sentence for sexual intercourse without consent until he was exonerated in 2002, the director of the Montana State Crime Lab told the jury that hairs found on a blanket in the victim’s house “matched” hairs taken from Bromgard’s body. There were so many hairs that matched so well, the analyst said, that there was a “one in 10,000” chance the hairs could have come from anyone else.

But no one has ever established any statistics about the microscopic characteristics of hair, so “one in 10,000” odds isn’t based on scientific consensus. How common is it for a person to have a particular hair color, or for a hair to crinkle or curl just so? Scientists have never answered that question systematically. And what does “match” mean, anyway? There are no uniform guidelines to say how many characteristics two hairs must have in common before they’re said to “match.” It varies entirely from one lab to the next, from one technician to the next.

Barry Fisher, who served as the crime-laboratory director for the Los Angeles Sheriff’s Department for more than 20 years, was often stymied by this problem when he took the stand. “How do you convey the level of certainty?” Fisher asks. “Do you say to the jury, ‘I’m pretty sure’? ‘I’m very sure’? What do these things mean?”

To get around this problem, Garrett found, forensics experts too often overreach in their testimony.

When Ray Krone was convicted of murder in Arizona and sentenced to death in 1995, the testimony of a bite-mark analyst was key to the state’s case. “This is really an excellent match,” the analyst said on the stand, comparing Krone’s teeth with a bite mark on the murder victim. “That tooth caused that injury.”

In fact, in its report the National Academy of Sciences found that, among all the forensic disciplines, only DNA has proved capable of “individualization”—that is, demonstrating “a connection between evidence and a specific individual or source.” When the DNA in the Krone case was tested year later, he was exonerated, but only after spending a decade in prison.

The report has led a small but growing number of judges to take a more skeptical approach to forensics. In addition to the Supreme Court case, Melendez-Diaz v. Massachusetts, U.S. District Court Judge Nancy Gertner announced in March that she will allow forensic evidence in her courtroom only if a lawyer first proves in a pretrial hearing that the method is scientifically sound. “In the past, the admissibility of this kind of evidence was effectively presumed, largely because of … the fact that it had been admitted for decades,” Judge Gertner wrote in her order. “The NAS report suggests a different calculus.”

The National Institute of Justice has funded some preliminary studies to establish the scientific information that has so far been missing; UCLA’s Mnookin and her colleagues are less than a year into a two-and-a-half-year grant to develop a more formalized and scientifically validated approach to fingerprint analysis. “It’s not that we know that they don’t work,” Mnookin says of fingerprints and other forensic methods. “It’s that we don’t have enough evidence about when they work, how they work, when they might not work.” The report also led to a series of Senate Judiciary Committee hearings. In January Senator Patrick Leahy (D-Vt.) introduced a bill to address some of the major issues in the nation’s forensic system. The The Criminal Justice and Forensic Science Reform Act takes up many of the issues identified in the NAS report. Although the report has gotten a chilly reception from many forensics experts and prosecutors, many others in the field, like Fisher, believe reforms in the system are long overdue.

Geoffrey Mearns, a former federal prosecutor who helped try both Oklahoma City bombers Terry Nichols and Timothy McVeigh, regularly used forensics in his work. Mearns served on the committee that wrote the academy’s report. “I had assumed that there were well-established uniform processes and procedures in place. I really had faith in the accuracy, reliability, and that it was well grounded in science,” says Mearns, now provost and senior vice president for academic affairs at Cleveland State University. “When I realized my faith was not well placed, I was very concerned about the damage that it was doing to the accuracy and efficiency of law-enforcement investigations. Because if the science is not accurate, and is leading us to the wrong person, it’s not only causing a terrible injury to the wrong person, but it’s leading you away from the right person.”

The 265 innocent people so far exonerated by DNA are lucky. Among the “hundreds, if not thousands” of people that the Innocence Project’s Peter Neufeld estimates were wrongfully convicted on the basis of faulty forensics, only a small percentage have DNA available to test. What is their recourse? Neufeld says his organization is counseling attorneys to submit a writ of habeas corpus—the legal system’s document of last resort—on the basis of newly discovered evidence: the fact that forensic science is not as scientific as it purported to be at the time of trial. However, “given the reluctance of judges to ever set aside convictions with anything less than DNA,” says Neufeld, “I am not as optimistic as I would like to be despite the fact that there’s a matter of fairness.”

One of those exonerated after 15 years in prison was Roy Brown. He was convicted of murder in 1992 and sentenced to 25 years to life, partly on the basis of a bite-mark analyst who said that Brown’s teeth matched a wound on the victim “to a reasonable degree of dental certainty.” The fact that whoever had bitten the victim had six teeth on his upper jaw—the wound clearly had six impressions—whereas Roy Brown had only four was “inconsistent,” the analyst admitted, “but explainably so in my opinion.”

DNA proved him innocent in 2006.