Conditional Probability


Conditional Probability
-----------------------



If A and B are DEPENDENT events.  The probability of B given A, 
P(B|A), is given by:

P(B|A) = P(A ∩ B)/P(A) ... 1.

Note:  If the events are INDEPENDENT then:

P(A ∩ B) = P(A).P(B)

Rearranging 1. we get:

P(A ∩ B) = P(B|A).P(A)

Bayes Theorem
-------------

From before we had:

P(A ∩ B) = P(B|A).P(A)

We can also write:

P(B ∩ A) = P(A|B).P(B)

Since P(A ∩ B) = P(B ∩ A),

P(B|A).P(A) = P(A|B).P(B)

Or,

P(A|B) = P(B|A).P(A)/P(B)

Where,

P(A) is the PRIOR, the initial probability of A.
P(A|B) is the POSTERIOR, the probability of A after B.
P(B|A)/P(B) is the support B provides for A. 

From the above Tree diagram we can see that:

P(B) = P(A ∩ B) + P(~A ∩ B)

     = P(A).P(B|A) + P(~A).P(B|~A)

Therefore,

P(A|B) = P(B|A).P(A)/{P(A).P(B|A) + P(~A).P(B|~A)}

We can generalize this to:

P(A₁|B) = P(B|A₁).P(A₁)/Σ_kP(B_k)

        = P(B|A₁).P(A₁)/Σ_kP(B|A_k).P(A_k)
  
        ≡ P(A₁ ∩ B)/Σ_kP(A_k ∩ B)

This is BAYES' THEOREM.

P(B) = Σ_kP(B_k) ≡ Σ_kP(B|A_k).P(A_k) ≡ Σ_kP(A_k ∩ B) is the LAW OF TOTAL 
PROBABILITY.

Bayes' Theorem calculates the probability that an event (A) occurs 
given knowledge of another event (B).  It can also be understood 
as a way of understanding how the probability that a theory is true 
is affected by a new piece of evidence.

It is used for many purposes, including detecting faults, surveillance, 
military defence, search-and-rescue operations, medical screening 
and even email spam filters.


Example 1:

Consider 5 marbles in a bag:  2 blue and 3 red.  What is the
probability of:

1.  Pulling 2 blue marbles without replacement?
2.  At least one blue marble being pulled?
3.  Pulling 2 blue marbles with replacement?
4.  The first marble being blue given that the second marble is 
    blue with replacement.
               
1.  These are DEPENDENT events.  To calculate the probability of
pulling a blue marble given that the first marble picked was blue,
we use the formula:

P(B₁ ∩ B₂) = P(B₂|B₁).P(B₁)



1 blue given 1 blue ( P(B₁ ∩ B₂) ):  (2/5)(1/4) = 1/10
                      
1 red given 1 blue ( P(R₁ ∩ B₂) ):   (2/5)(3/4) = 3/10
                                
1 blue given 1 red ( P(B₁ ∩ R₂) ):   (3/5)(2/4)^* = 3/10
      
1 red given 1 red ( P(R₁ ∩ R₂) ):    (3/5)(2/4) = 3/10

^* Remember a blue marble wasn't picked so there must still be 2.

The probabability of 2 blue is 1/10.  

2.  The probability of at least 1 blue is 1/10 + 3/10 + 3/10 = 7/10.

3.  These events are INDEPENDENT and we get:

1 blue given 1 blue ( P(B₁ ∩ B₂) ):  (2/5)(2/5) = 4/25
                      
1 red given 1 blue ( P(R₁ ∩ B₂) ):   (2/5)(3/5) = 6/25
                                
1 blue given 1 red ( P(B₁ ∩ R₂) ):   (3/5)(2/5) = 6/25
      
1 red given 1 red ( P(R₁ ∩ R₂) ):    (3/5)(3/5) = 9/25

The probabability of 2 blue is 4/25.  The probability of at least 
1 blue is 4/25 + 6/25 + 6/25 = 14/25. 

4.  To calculate the probability that the first marble was blue 
given that the second marble picked was blue, we use Bayes' 
formula:

P(B₁|B₂) = P(B₂|B₁).P(B₁)/P(B₂)  

Where,

P(B₁) is the PRIOR, the initial probability of B₁.
P(B₁|B₂) is the POSTERIOR, the probability of B₁ after B₂.
P(B₂|B₁)/P(B₂) is the support B provides for B₁. 

But P(B₂) = 1/10 + 3/10 from answer 1.  Therefore,

P(B₁|B₂) = (2/5)(1/4)/{1/10 + 3/10}   ... dotted red lines
 
       = 1/4

So the probability that a blue was picked initially given that a 
another blue was picked the second time is 'modified' from
the prior value of 2/5 to 1/4!

Example 2:

1% of women have breast cancer.  A woman with breast cancer 
has a 90% chance of testing positive while a woman without
has a 5% chance (false positive).   What is the probability a
women has breast cancer given that she had a positive test?

P(C|+) = P(+|C).P(C)/P(+)



+ given cancer ( P(B ∩ B) ):  (0.01)(0.90) = 0.0090
                      
- given cancer ( P(R ∩ B) ):   (0.01)(0.10) = 0.0010
                                
+ given no cancer  ( P(B ∩ R) ):   (0.99)(0.10) = 0.0990
      
- given no cancer ( P(R ∩ R) ):    (0.99)(0.9)  = 0.8910

P(C|+) = 0.0090/{0.0090 + 0.0990}

       = 0.0090/0.1080

       = 0.08333

So the probability of having cancer given that the test is 
positive is actually quite low!.

Redshift Academy