Bayes' Theorem - Introduction

Lawyers often talk about Bayesian analysis. It is, in fact, one of the major ways in which maths play a role in law: it is used for estimating the probative value of certain pieces of evidence in the presence of other evidence.

We will want to refer to Bayes' theorem quite often, so this post is devoted to a simple and complete explanation.

First of all, let’s introduce some notation. We write:

P(A) = probability of A = probability that the event A happens

P(A|B) = probability of A given B = probability that the event A happens, given that the event B has happened.

Bayes’ theorem relates P(A|B) to P(B|A). And it turns out that the relation involves what is called the prior probabilities – the probabilities P(A) (probability that A happens, without considering B at all) and P(B) (probability that B happens, without considering A at all).

The exact relationship is the following:

P(A|B) = P(B|A) * P(A) / P(B)

Now, this may not mean anything to you right now, but it is actually very counter-intuitive. People generally feel like they have a good idea of what P(B|A) is if they know P(A|B) and just one of the prior probabilities, like P(B). But depending on what the prior probability P(A) is, that good idea may in fact be completely off.

For example, imagine you are a teacher, and you give your students a very difficult test, which every year only 5% of your students get an A on. Obviously a good (or bad) way for a student of increasing his chances is cheating – say that cheating gives him a 40% chance of getting an A. Now, imagine that you have a student who has got an A. As the teacher, you might be tempted to think that the student has cheated. Well, Bayes’ theorem tells you that you cannot know how likely it is that this is the case if you don’t consider the prior probability of cheating.

First, let’s figure out the information we do have:

Call A the event getting an A on the test, and B the event that the student has cheated.

Then we know P(A|B) = 0.4 and P(A) = 0.05.

We want to know P(B|A), the probability that the student has cheated, given that he got an A.

Since we need to know the probability of cheating, P(B), let's consider two different situations.

First situation: suppose this was a fairly relaxed test setting, where students were not supervised very carefully, and you estimate that 10% of them cheated, ie P(B) = 0.1.

Then Bayes’ theorem tells us that P(B|A) = P(A|B)*P(B)/P(A) = 0.4*0.1/0.05 = 0.8. In other words, if the student got an A, then there's an 80% chance that he did it by cheating.

Second situation: suppose this was a very controlled exam, where students were in individual rooms with individual supervisors. The chance of cheating is still not 0, but it’s definitely a lot smaller, say just 1%, or 0.01. Then Bayes’ theorem tells us that P(B|A) = 0.4*0.01/0.05 = 0.08, or 8%. So if you have a student who got an A, it's not that likely that he cheated, and you need not be too concerned.

Of course, this is a rather simple problem – it is a lot more probable that the student cheated if he was in an environment where it was easy to cheat, than if he wasn’t – not exactly surprising. But the problems can get a lot more confusing than that, as we will see next time!