How Experts in Forensic Document Analysis Combat Cognitive Bias

In forensic science, subjective judgments will always play a role in how experts analyze and interpret impression evidence. Although subjective judgments are not wholly problematic, they increase the likelihood for bias when additional contextual information is introduced—even if this information is not entirely relevant to the task. This is particularly important in the forensics field, where an examiner’s findings can greatly impact the outcome of a criminal or civil case.

Forensic scientists like to believe they are immune to bias and safeguarded by the strength of their rigorous training programs. However, bias may still affect many forensic impression disciplines. Some of these disciplines include firearms and tool marks, fingerprints, documents, and tire impression evidence.

This article explores how cognitive biases can affect the decision-making process in forensic examinations and strategies that experts in forensic document analysis use to mitigate irrelevant or biasing information in handwriting examinations.

Understanding Human Biases

Human biases govern how we make judgments and decisions in our lives. But in the legal system, it’s crucial to understand how these patterns of information processing can influence forensic practitioners in their evidentiary examinations. Two main forms of bias that may affect forensic professionals include cognitive and contextual biases.

1) Cognitive Bias

Cognitive bias is a systematic pattern of deviation in human judgment. Humans are predisposed to take mental “shortcuts” when faced with complex cognitive problems. By utilizing these shortcuts, a vast amount of information can be processed quickly without taxing the brain. Cognitive bias tends to occur when a decision is made under free choice and the person is committed to their decision. When these conditions are met, individuals will actively seek information that bolsters their view or produces easily refutable findings.

Also referred to as selective attention, an example could be errors in memory that impact how or what you think about a particular task or event. This, in turn, will influence how you think about similar events. In another form of cognitive bias, selective information seeking describes a scenario where individuals seek out information that easily disconfirms alternative explanations.

2) Contextual Bias

Contextual bias is knowledge (which can be relevant or irrelevant) concerning a particular fact or circumstance relating to a case or examination that can impact objectivity. Consider, for example, a document examiner conducting a signature examination with three possible writers. During the examination of the signature, it is revealed that one of the writer’s latent fingerprints has been matched to the document in question. Though this new piece of information is not relevant to handwriting analysis, it does present a possible influence on the examiner’s final conclusion—and the case at hand.

Exploring Biases in Forensic Handwriting Analysis

Industry governance has also taken notice of possible biases within the forensics field. In a joint effort from the National Institute of Standards and Technology (NIST), a federal agency overseeing science, innovation, and technology, and the National Institute of Justice (NIJ), the research, development, and evaluation agency of the Department of Justice, a working group was created. This group called the Expert Working Group for Human Factors in Handwriting Examination, set out to explore human factors as it relates to other elements of a system—technology, training, decisions, products, procedures, workspaces, and the overall environment— through scientific assessment detailing the effects on forensic document examinations.

The Working Group adapted a graphical model from Dr. Itiel E. Dror, Ph.D., a renowned cognitive neuroscience expert, to understand how different sources of contextual information may affect forensic casework. Starting with the Questioned Document itself, the Working Group identified seven levels of information exposure that can lead to bias and how to help mitigate these potential negative effects.

1) The Questioned Document

The content and meaning of the written words that are being analyzed can form bias for the examiner. For example, the examiner is conducting analysis on a detailed diary of sensitive and personal information purportedly written by a suspect in custody. Bias could form after reading information concerning financial hardships in a case where the suspect is being investigated for embezzlement.

2) The Known Writing

The reference material used to compare against the questioned document could form bias when an examiner is reviewing and extracting meaning from course-of-business documents and personal correspondence (e.g. diaries, notebooks, letters, and cards).

3) All Other Information

This includes any other oral, written, or behavioral information that is not directly part of the questioned or known writing. This information could be communicated through attorneys, police officers, or through written reports of other experts, oral discussions, and exchanges by email or other means.

4) An Expectation of Outcome

Examiners may have their own expectations about the outcome of a case before it’s been analyzed that can contribute to examination bias. For example, an examiner could conclude that since a sample is being sent to a federal forensic laboratory there must be an underlying reason linking the suspect and the handwriting. In this scenario, the examiner has an expectation that the evidence submitted is inculpatory before it’s even reviewed.

5) The Laboratory’s Organization and Culture

Creating a positive experience and perception for employees within a lab is multifaceted. A constructive lab environment includes open communication, appropriate training, qualified trainers, robust quality assurance, external accreditation, continued professional development opportunities, a well-designed laboratory, promoting positive error culture, and transparent management. While this list is not exhaustive, these factors go a long way in creating a healthy workplace culture. For labs failing to foster a supportive environment, they risk producing a sloppy work product due to lack of protocols and procedures, low morale due to lack of educational opportunities for professional growth, and the possibility of falsifying data and evidence for fear of management’s reprisals.

6) The Examiner’s Training and Motivations

Good training alone cannot shield an examiner from biasing information. However, a competence-assessed modular training program, including training in cognitive bias, could provide trainees with an awareness of this weakness in human performance. Equally important would be developing a Contextual Information Management system (CIM) limiting potentially biasing information in policy and procedure manuals. Clear, direct communication from leadership about these policies and work quality expectations also aid in promoting positive laboratory culture and better-quality work.

7) The Brain and Human Factors

As humans, we’re already predisposed to create bias simply by what we see in the world around us. Therefore, the cognitive architecture of the brain and its connection to all human factors will also contribute to forensic examination bias.

Ways to Limit Cognitive Bias in Expert Examinations

Efforts to limit contextual bias are gaining traction as a viable option for casework within laboratories. This is also a valuable step to take as an attorney contacting potential forensic experts. In working with forensic document examiners (FDE), attorneys should develop agreements outlining what information is relevant to the task at hand. This kind of planning before collaborating on a handwriting analysis project will aid in avoiding contextual bias.

In larger laboratory settings, administrative staff can also assist through Contextual Information Management (CIM) systems. A CIM can help control which information reaches the examiner and limit potential biasing information. As the study of bias within forensic handwriting examination is ongoing, information is still being gathered on many more potential sources of contextual information. This will be an area to watch for the latest innovations for future collaborations between forensics and legal professionals.

References

McClelland, J. and D. Rumelhart 2011. “An interactive activation model of context effects in letter perception: Part 1, an account of basic findings.” Psychological Review 88(2): 375; Wilson, T., and N. Brekke.1994 “Mental contamination and mental correction: Unwanted influences on judgments and evaluations.” Psychological Bulletin 116 (1):117-142.

Festinger, L. 1957 A Theory of Cognitive Dissonance. Stanford, CA: Stanford University Press.

National Institute of Science and Technology (NIST) “Forensic Handwriting Examination and Human Factors: Improving the Practice through a Systems Approach” February 2020. 30-39