At the recent Texas Forensics Science Seminar sponsored by the Criminal Justice Integrity Unit of the Court of Criminal Appeals, I attended presentations by DPS crime lab experts on ballistics, toolmarks, hair (and other trace) evidence, and fingerprint examination. Much of the discussion at the event was framed by the 2009 report (pdf) from the National Academy of Sciences which was particularly hard on forensic disciplines which rely upon subjective comparisons, from fingerprints to hair comparisons to shoeprints. After hearing DPS’ own experts discuss how they perform these tasks, it’s difficult not to agree that the basis for these practices is not scientific in the same sense as, say, DNA analysis.
Dr. Jay Siegel, who was one of 17 people on the NAS panel that produced the report, told the seminar audience to closely examine terminology from experts providing so-called “pattern evidence”: Ask them, what is the definition a “match”? By what methodology can the examiner individualize their findings? And beware “weasel words” like “consistent with,” he said, or statements that an item “could not be eliminated as the source.” Later in the day, however, the ballistics/toolmarks expert who presented on those subjects for DPS admitted he used those phrases in his work and defended doing so. (This despite the fact that even bullets fired from the same gun may have different markings, he said.) For these disciplines, the NAS’ critique that they’re subjective comparisons, not science, was impossible to avoid. That’s true even (perhaps especially) regarding fingerprints, which for generations has been considered the gold standard of modern forensics. Crime lab experts are able to make these distinguishing judgments, we were told, based on their “training and experience,” but when the process was described in detail, it all came down to “After looking at them closely, I think these two things look alike.” There’s an argument to be made that that should be enough, that forensic testimony shouldn’t “guild the lilly” by claiming experts can individualize their findings to a greater extent than is scientifically valid.
Even more subjective, and typically performed by poorly trained police officers in the field at DWI stops, is the Horizontal Gaze Nystagmus (HGN) test, where officers are told to hold a flashlight or other object at the edge of the driver’s peripheral vision, move it horizontally back and forth, and look for minute cues regarding their eye movements that weren’t even obvious on the training video. The DPS trainer who gave the seminar presentation readily granted that officers who didn’t routinely perform the test and stay up on their training often performed the procedure incorrectly or misinterpreted the results, though he defended the test when performed properly. Either way, it couldn’t be a more subjective technique, even when performed correctly. Another speaker from the American Academy of Forensic Sciences said in his opinion HGN shouldn’t be considered “science,” though courts routinely accept its results as probable cause in DWI cases.
When you add in all the subjective disciplines to other forensic methods with gaping methodological flaws, the list of forensic sciences that aren’t really science grows quite long. The following top-ten list isn’t comprehensive – just the first ten that came to mind – and I encourage readers to add to it in the comments.
Top Ten Forensic Sciences That Aren’t Science
1. Fingerprint examination
2. Horizontal Gaze Nystagmus (field sobriety test)
3. Ballistics
4. Blood splatter
5. Hair/trace-evidence comparisons (non-DNA)
6. Toolmarks
7. Polygraphs
8. Arson (older cases)
9. Autopsies
10. Dog-scent lineups
I’m not arguing that these forensic disciplines are always wrong; in fact, subjective comparisons by a person (or perhaps even a dog) who performs the task frequently may usually turn out to be accurate. But that doesn’t make what’s going on scientific or really more than an educated opinion as opposed to a definitive, science-based finding of the type associated with DNA or identification of controlled substances. It seems unwise to overstate conclusions from those disciplines when their scientific basis has been so widely and publicly questioned and the needed research to validate them hasn’t been done.
