2.3 Uncertainty in scientific evidence
There are multiple sources of aleatory and epistemic uncertainty in forensic scientific evidence. As well as the unique uncertainties that arise from individual case scenarios, there are generic uncertainties which relate to the collection, analysis, and interpretation of different scientific evidence types. Some of these generic uncertainties are shared across evidence types, for example the transfer and persistence of materials on surfaces, whilst others are specific to a particular evidence type, such as the analysis of low template or mixture DNA evidence. Here we consider general uncertainties of the forensic scientist and how they can be approached.
Consider the following questions when thinking about uncertainty in forensic scientific evidence:
- What are we uncertain about?
- What are the sources of uncertainty?
- What is the level of our uncertainty?
- What is the magnitude of uncertainty?
- In what form is the uncertainty communicated?
These questions follow the model of communicating uncertainty presented in Bles et al. (2019) and we will apply this model to forensic evidence.
What are we uncertain about? This is the object of uncertainty. This is commonly the sequence of events which led to the evidence. Nested within this, there can also be other objects such as the characteristics of the evidence, the analysis of the evidence, comparisons of the evidence to information within databases, expert opinion concerning the evidence, etc.
What are the sources of uncertainty? These are the reasons for uncertainty. Imprecise measurements, contaminated evidence, lack of scientific knowledge, database limitations, subjective interpretation, etc.
These first two questions make it clear exactly what we are uncertain about and why. In an expert witness report, they can appear as the rationale behind what the expert has been tasked to do and why they have done it.
What is the level of our uncertainty? This is whether the uncertainty relates directly to the object itself or indirectly through uncertainties about things which affect the object. For example, uncertainty about an item of evidence is direct but lack of confidence in the science underlying the evidence interpretation is indirect.
This particular question is important because these two levels of uncertainty can be evaluated differently. Direct uncertainty is often easier to quantify than indirect uncertainty, which will be explained in more detail in the next chapter. In practice, this means that in expert witness reports it is easier to treat direct uncertainties with an evaluative likelihood ratio approach than it is for indirect uncertainties. Indirect uncertainties may be listed in the expert’s report as a verbal caveat for the expert’s evaluation or may have contributed to a decision not to evaluate the evidence using a likelihood ratio approach.
What is the magnitude of uncertainty? This is the extent or size of how uncertain we are. Does the measurement technology have a high or low level of precision? Is the database highly reliable or not? Is the expert’s interpretation more or less reasonable compared to other experts in their field? How well does the evidence support the prosecution?
In what form is the uncertainty communicated? What tools are used to communicate the uncertainty to others. Is uncertainty mentioned in the expert report? Is it clear that uncertainties have been addressed or are they only acknowledged? Has a number been used to convey the uncertainty, or a verbal expression?
The final two questions above are examples of evaluating and communicating uncertainty. In expert witness reports, this is where the expert presents their evaluative opinion having considered the evidence alongside the case circumstances and hypotheses put forth by legal counsel. If they have used a likelihood ratio approach, then this evaluation will be given on a numerical or verbal scale (in the UK). For certain types of evidence such as fingerprint identification, the expert may give their opinion as the inclusion or exclusion of an individual.
Forensic science is a continuously developing field. Expert interpretations which historically overlooked (and in some cases ignored) major elements of uncertainty in their discipline have been and continue to be updated by practices which are informed by scientific evidence. However, there is still a long way to go in some fields e.g. understanding transfer and persistence of fibres.
There will always be aleatory uncertainty due to the random nature of real-world processes and epistemic uncertainty due to the limitations of our knowledge. This explains reasonable disagreement between experts in which personal uncertainties may diverge. These disagreements can still be settled in the mind of the fact finder in the same way as disagreements in other aspects of evidence: through cross-examination.