Should Expert Witnesses Always Rely on Peer-Reviewed Methodologies?

Currently, the states are split, to various degrees, between the two major competing standards: Frye v. United States, 293 F. 1013 (D.C. Cir. 1923), and Daubert v. Merrell Dow Pharmaceuticals, Inc., 509 U.S. 579 (1993).

Under Frye, an expert’s opinion must be “generally accepted” within the scientific community, whereas under Daubert, general acceptance is one of several factors to consider. Though notably different, each standard employs, to a certain extent, the concept of peer review: that is, the review of an expert’s research by members of its own community prior to publication. In Daubert, whether an experts’ technique or theory has been subject to peer review and publication is a clearly enumerated factor. And while not explicitly stated in Frye, the prevalence and significance of peer review can affect a court’s decision when determining if an opinion is generally accepted in the scientific community.

However, the weight given to peer-reviewed opinions is not a bright-line rule. Depending upon the evidentiary standard, and the offered expert opinion, the importance of peer review varies when assessing admissibility.

What is Peer Review?

Peer review is the evaluation of a scientific work by others that are experts within the same field. The purpose of peer review is to ensure that only valid, reliable research is published. The reviewers of such research provide suggestions to the authors regarding how to improve the quality of their work before it is published in any scientific journal or scholarly publication.

The peer-review process has a long history, as it was first used as a method of evaluating written work in ancient Greece. With the advent of the printing press in 1453, which allowed the distribution and dissemination of written documents to the general public, peer review became a way to ensure and regulate the quality of a scientist’s work. Peer review is now the standard practice of most scientific journals.

The peer-review process begins when a scientist (or another specialized professional) completes research and writes a manuscript describing the purpose, methodology, and conclusions of the study. The author then submits the manuscript to a relevant journal within their specific field. At which point, the journal’s editors review the manuscript. Then, if acceptable, the manuscript is sent to other reviewers, also known as referees, for evaluation of its findings. The reviewers analyze the validity of the science, the quality of the study’s design, and the appropriateness of the used methodology. The reviewers are also tasked with identifying any errors in the work, which are then passed to the author to fix. They then offer a recommendation to the journal’s editor regarding whether the manuscript should be accepted. One study has found that approximately 50% of submitted manuscripts are accepted for publication, with 41% requiring revisions prior to acceptance.

Peer review is generally conducted in one of three ways – open review, single-blind review, or double-blind review. In open review, the author and reviewer are aware of each other’s identities. In double-blind review, both are unknown to the other. In a single-blind review, the author is known to the reviewer but the reviewer’s identity is kept secret. Each has its own advantages, but single-blind review is most common, with approximately 85% of authors being subjected to this kind of review. The purpose of any blind review is to prevent bias and increase the quality of the review.

The major criticism of peer review is that there is little evidence to support the belief that the process works, with one 2002 study conducted by the Journal of the American Medical Association concluding that peer review is “largely untested and its effects are uncertain.” Critics also argue that the peer review process is not thoroughly conducted and does not effectively detect errors. Some researchers have complained that the process stifles their work, since innovative idea may be looked upon less favorably in the review process. In addition, there is also the question of reviewer competency and whether the reviewer possesses the requisite and relevant expertise.

Peer Review under Frye

Unlike Daubert, peer review is not an enumerated factor that must be considered under the Frye standard. The inquiry under Frye is way more pointed – is this expert’s opinion generally accepted within the scientific community? It is also important to note that Frye only applies to novel scientific evidence, not established principles, for which there is no threshold inquiry. That does not mean that peer review in unimportant. In the quest to prove general acceptance, scientific publications may be considered. However, in light of the arduous process of peer review, it may take a novel scientific idea quite sometime before it attains publication. Frye also permits testimony from scientists in order to establish the acceptance of the expert’s opinion, which in some instances, may circumvent the peer review process.

That being said, publication and peer review have long been hallmarks in determining expert admissibility under Frye. In certain cases, the offered opinion required the court to address whether the methodology had been subject to sufficient peer review. See United States v. Brown, 557 F.2d 541, 557 (6th Cir. 1977) (excluded testimony regarding ion microprobic analysis because the experts were unable to cite any authority to the field to support their positions); See also United States v. Zeiger, 350 F. Supp. 685, 689 (D.D.C. 1972) (relied on published studies to support the admission of polygraph examination results).

What Does Daubert Say?

Under Daubert, relevance and reliability are the focal points when admitting expert testimony. Placing an emphasis on the court’s “gatekeeping responsibility,” Daubert provides a non-exhaustive list of factors to consider such:

1. Whether the expert’s technique or theory can be tested and assessed for reliability
2. Whether the technique or theory has been subject to peer review and publication
3. The known or potential rate of error of the technique or theory
4. The existence and maintenance of standards and controls
5. Whether the technique or theory has been generally accepted in the scientific community

So where does peer review fall? No one factor is given more weight than another, so the presence (or absence) or peer review is not dispositive. As Daubert acknowledges, “in some instances well-grounded but innovative theories will not have been published…Some propositions, moreover, are too particular, too new or too limited interest to be published.”

Like most inquiries under Daubert, the analysis is fact-specific. For example, a lack of peer review or publication is deemed unimportant when the opinion is supported by widely accepted scientific knowledge. See Kannankeril v. Terminix, International, Inc. 128 F.3d 802, 809 (3d. Cir. 1997). On the other hand, since Daubert has been expanded to include non-scientific experts (See Kumho Tire Co. v. Carmichael, 526 U.S. 137 (1999)), who rely on “skill- or experienced-based observations,” i.e., law enforcement experts, peer review (though still a factor to consider) may sometimes be unavailable in these instances.

Overall, the purpose of peer review is to ensure the reliability and validity of an expert’s methodology. However, the limitations and flaws within the process should not be overlooked.