Statistical and Causal Fallacies in Court: A Lawyer’s Practical Guide

Statistical and causal evidence can assist a court, but neither a probability nor an association is self-explanatory proof. A reliable argument identifies the proposition to be proved, the population or data set used, the assumptions behind the calculation, the limits of the evidence and the alternative explanations that remain. The prosecutor’s fallacy, base-rate error, selection bias and confusion of correlation with causation are dangerous because they can make a modest inference sound certain. Lawyers should translate numerical evidence into the legal question without overstating what the numbers establish.

Start with the proposition the court must decide

Before discussing a percentage, state the precise legal and factual proposition. Is the issue identity, causation, quantum, discrimination, reliability of an expert method or likelihood of a future event? The same statistic can have different significance depending on the question. A probability that a profile occurs in a population is not automatically the probability that a particular person is responsible. An observed association is not automatically proof that one event caused another.

Separate the evidence from the ultimate conclusion. The evidence may show that a condition is more common among one group, that a document pattern is unusual, or that an event followed another. The legal conclusion requires further steps: relevance, reliability, competing explanations, burden and standard of proof. State those steps rather than hiding them in a number.

Avoid the prosecutor’s fallacy and base-rate errors

The prosecutor’s fallacy confuses the probability of observing evidence if a person is innocent with the probability that the person is innocent given the evidence. Those are different propositions. A rare matching feature may be relevant, but its force depends on the reference population, the number of possible sources, laboratory or collection error, and the other evidence in the case.

Base rates matter because a striking percentage can mislead where the underlying event is rare. Ask how many people are in the relevant population, how many could have produced the observation and how the sample was selected. A lawyer should not invite the court to treat a statistical figure as a direct measure of guilt, liability or causation unless the method actually supports that inference.

Distinguish correlation from causation

Correlation means two variables move together; it does not by itself establish why. A loss may follow a breach because of market conditions, another actor, a pre-existing weakness or a combination of causes. A workplace outcome may be associated with a policy because of selection, timing or another unmeasured factor. The causal question requires a mechanism, chronology, comparison and consideration of alternatives.

Test the proposed causal chain. What act is said to have produced what result? Was the result foreseeable? What independent event may have intervened? Is there evidence from records, expert analysis or a counterfactual comparison showing what would probably have happened otherwise? The legal test for causation varies by claim, but unsupported sequence is rarely enough.

Interrogate samples, methods and expert assumptions

Statistics are only as useful as the underlying data. Ask who was included, who was excluded, how observations were gathered, whether the sample represents the relevant population, and whether the method was applied consistently. A small or selected sample may be useful for a limited proposition but unable to support a broad conclusion. Missing data, changed definitions and untested assumptions should be disclosed rather than treated as noise.

Expert evidence requires the same discipline. Identify the expert’s instructions, expertise, materials reviewed, method, margin of uncertainty and any assumptions supplied by the instructing party. Cross-examination or written challenge should focus on the inference the expert asks the court to draw, not merely the presence of technical language. An expert may be qualified and still rely on a method that cannot answer the legal question posed.

Present quantified evidence with disciplined language

Use words that match the proof. “Consistent with”, “associated with”, “increases the likelihood” and “subject to stated assumptions” may be accurate where “proves” is not. Explain percentages in ordinary terms and identify the denominator. If a chart, table or calculation is relied on, make sure the court can trace its source and method. Apparent precision should not conceal uncertainty.

Address contrary material candidly. If another data set, expert opinion or explanation points in a different direction, state why it is less reliable, less relevant or incapable of displacing the preferred inference. Do not imply that uncertainty disappears because the preferred conclusion is commercially attractive or emotionally compelling.

Use a final inference check before filing

For every quantified argument, ask: what does the figure actually measure; what proposition does it not measure; what assumptions connect it to the legal issue; and what alternative explanation remains? Then check whether the pleading, affidavit and submissions use the same careful formulation. This is general information, not advice on a particular proceeding.

Comparisons require an appropriate control. If a party says an outcome is unusually frequent, ask compared with whom, during what period and after accounting for relevant differences. A comparison between unlike groups can create a false impression of discrimination, loss or risk. A comparison using a carefully defined reference group may be probative, but its limits should be stated.

Regression, modelling and forecasting deserve particular caution. A model can organise information and estimate a range; it cannot remove uncertainty created by missing variables, subjective choices or unstable assumptions. Identify the input data, the variables excluded, the period covered and how sensitive the conclusion is to a change in assumptions. The court should be told whether the model describes past data, predicts future outcomes or merely illustrates one scenario.

Quantified damages require the same rigour. A projection of lost profit, value or future expense should disclose the starting figures, assumptions, discounting, contingencies and alternative scenarios. The opposing party should be able to test the arithmetic and the legal relevance of each assumption. A precise total is not necessarily a reliable one if the premises are speculative.

When challenging statistics, do not imply that all numerical evidence is worthless. Identify the specific defect and its consequence: an unrepresentative sample may limit generalisation; an unexplained denominator may distort frequency; an omitted variable may weaken causation; and an unsupported expert assumption may reduce the weight of the opinion. This focused approach helps the court decide what weight, if any, the evidence should receive.

Finally, preserve the underlying data, calculations and instructions where disclosure rules require it. An attractive summary cannot be fairly evaluated if the other party cannot inspect the material from which it was derived. Transparency improves both the reliability of expert evidence and the quality of legal submissions built upon it.

In oral argument, slow the inference down. State the observed fact, the calculation or expert conclusion, the assumption that connects it to the disputed issue, and the legal consequence sought. This sequence allows the court to accept a reliable part of the evidence without being pressed to adopt an overbroad conclusion. It also exposes whether the disagreement is about data, method, interpretation or the governing legal test.

The same discipline protects both sides from treating uncertainty as proof. It enables the court to give reliable quantitative evidence appropriate weight while rejecting conclusions that travel beyond the data, method or legal issue presented.

Official source: Constitution of Kenya, 2010 (Kenya Law).