What are the implications of a court banning Bayesian probability?

In a recent judgement the English Court of Appeal has not only rejected the Sherlock Holmes doctrine shown above, but also denied that probability can be used as an expression of uncertainty for events that have either happened or not.

Court of Appeal bans Bayesian probability (and Sherlock Holmes)


The court is probably doing something useful. While Bayesian methods are the foundation of thinking and reasoning about the world, we do these types of inferences intuitively. Paradoxically, because our intuition for this is so developed, the mathematically untrained person can do it better without any math than with!

So for example, suppose I see find on a murder scene a red cap, a Camel cigarette, a shoe-print of size 14, and a blurry picture which shows the guy is 6'4. I can say in court, as a prosecuter: 4% of people smoke Camels. Only 8% of men have size 14 shoes, only .1% own a red cap, and only .1% of men are 6'4! Multiply, and you see the probability is 1 in a million. This is your guy!

It's very hard for a defense attorney to argue with this, even though we all know in our gut that this crap, that the evidence above is ridiculously weak. It is hard to explain why it is ridiculously weak without a long explanation of the selection process, the biases for keeping certain evidence and not others, and the ability of police to attach irrelevant stuff to the event that match, after finding the suspect. All these things render the 1 in a million more like 1 in 2, or 1 in 3.

Here is you see the main problem with Baysian reasoning in a courtroom (rather than a science laboratory), the method is being applied with a political end, and so it is not done honestly. The right way is to say how you selected the evidence you are presenting, and how you found your suspect. If you found your suspect from a million follks, by matching these traits, you have no evidence at all. This is exactly what a jury's intuition is from seeing this ridiculous evidence: you don't know anything. But using Baysian multiplication (inappropriately) a lawyer can try to persuade the jury that the evidence is much much better, since only about 5 pieces of evidence each with 10% prevalence and which match to the suspect are required for scientific certainty, or certainty beyond reasonable doubt. Yet precisely these kinds of vague-evidence are the easiest to spuriously attach to a case.

So the use of Bayesian probability in courtrooms is almost always a way of lying with statistics, a way of making weak evidence seem stronger by multiplying likelihoods inappropriately. It's another version of the conspiracy theorist's "What are the chances of THAT??" Often the chances are very good, because THAT is very fungible, it could be a billion coincidences.

The fact is that these probabilities are very hard to estimate by folks unskilled at statistics, and the manipulation of the statistics by attorneys is easy to do and hard to counter.

So I would argue that if you have good evidence, present it in such a way that the evidence looks good intuitively as well. I agree with this judge's judgement, or rather, I defer to their experience.