Redshift Academy


Sampling Distributions
----------------------

If we draw a sample size, n, from a given population and compute 
a statistic (mean, standard deviation, proportion) for each sample. 
The probability distribution of the statisitic is called a sampling 
distibution.

Central Limit Theorem
---------------------

For large enough sample sizes^* the sample MEANS follow N(μ,σ/√n) 
where n is the sample size.  In other words, the mean of all
the sample means is equal to the population mean.  This applies 
regardless of the distribution of the parent population.  As the 
sample size is increased the standard deviation, skew and kurtosis 
of the sampling distribution decreases. 

The SD of the sampling distribution of the mean is called the 
STANDARD ERROR OF THE MEAN. 

SE = σ/√n = σ_{sample means}

We can generalize this to:

SE = (σ/√n)((N - n)/(N - 1))

Where N is the population size.

For infinite N this reduces to:

SE = σ/√n as before

For small N it reduces to:

SE = σ

* A sample size ≥ 30 is generally considered to be the minimum 
  for the CLT to apply.  

Distribution of Means
---------------------
   _
If x is the mean of a random sample of size n taken from
a normal population having mean μ and variance σ², then
we can state the following:

Large Sample Size (n > 30)
--------------------------
      _
z  = (x - μ)/σ/√n  

Generally, the population standard deviation is not given. 
Under these circumstances it is necessary to compute it from 
the sample using:
      _
s² = Σ_i(x - x_i)²/(n - 1)

Why use n - 1?  To answer that we need to look at biassed
versus unbiassed estimators.

Biassed versus Unbiassed Estimators
-----------------------------------

Consider a small distribution:  5 7 12
N = 3
n = 2
μ = 8
σ² = Σ(x - μ)²/N = 8.67
σ = 2.94
           _                  _
s² = Σ(x - x)²/(n - 1) = Σ(x - x)²/1
           _
Sample     x      s²     s    s²/σ² 
------    ---   ----   ----   ----  
5   5     5.0    0.0   0.00   0.00    
5   7     6.0    2.0   1.41   0.23    
5  12     8.5   24.5   4.95   2.83        
7   5     6.0    2.0   1.41   0.23     
7   7     7.0    0.0   0.00   0.00
7  12     9.5   12.5   3.54   1.44
12  5     8.5   24.5   4.95   2.83
12  7     9.5   12.5   3.54   1.44
12 12    12.0    0.0   0.00   0.00

Average   8.0    8.67  2.20

Notes:
  _
- x follows a t distribution (normal dostribution if the 
  sample size is large).
- s is a biassed estimator of σ
- (n - 1)s²/σ² follows the χ² distribution.
- A denominator of n gives the biassed estimator for σ².
- A denominator n - 1 gives the unbiassed estimator for σ².

Small Sample Size (n ≤ 30)
--------------------------

If n ≤ 30 then we must use the t-statistic instead:
     _
t_v = (x - μ)/s/√n  

with v = n - 1 degrees of freedom.

The t-distribution has the following properties:

mean:  μ = 0 
variance:  σ² = v/(v - 2), where v is the degrees of freedom 
           and v > 2

The variance is always greater than 1, although it is close 
to 1 when there are many degrees of freedom. With infinite 
degrees of freedom, the t distribution is the same as the 
standard normal distribution.

Distribution of Proportions
---------------------------

If p is the proportion (probability) of successes in the 
population, then:

σ_p = √(p(1 - p)/n)

The sampling distribution of p is a discrete rather than a 
continuous distribution.  It is approximately normally 
distributed if n is fairly large and p is not close to 0 or 1.  
A general rule of thumb is that the approximation is good 
when: 

np and n(1 - p) are both ≥ 10
     _
z = (p - μ)/σ/√n  

As before, generally, p and σ are not given for the original 
population.  Under these circumstances it is necessary to 
compute them from the sample.  Thus,

p = p_sample

Distribution of Variances
-------------------------

If s² is the variance of a random sample of size n taken from 
a normal poulation having the variance σ², then

χ² = (n - 1)s²/σ²

with v = n - 1 degrees of freedom.

The χ² distribution has the following properties:

mean:  μ = v 
variance: σ² = 2v

Example:

An optical firm purchases glass to be ground into lenses and 
past experience has shown that the variance of the refractive 
index = 1.26 x 10^-4.  A shipment is received and a sample of 
20 pieces is pulled.  The measured variance of the sample is 
2.10 x 10^-4.  Should the sample be rejected?

H₀:  σ² = s²
H₁:  σ² ≠ s²

χ² = (20 - 1)2.10 x 10^-4/1.26 x 10^-4 = 31.66

From tables, χ²_0.05 for v = n - 1 = 19 is equal to 30.144.  
Since 31.66 > 30.144 the result is significant at the 0.05 
level and there is sufficient reason to reject H₀.

F Distribution
--------------

If s₁² and s₂² are the variances of independent random samples 
of size n₁ and n₂ taken from two normal populations having the 
same variance, then

F = s₁²/s₂²

with v₁ = n₁ - 1 and v₂ = n₂ - 1 degrees of freedom.

The F distribution has the following properties:

mean:  μ =  v₂/(v₂ - 2) for v₂ > 2.
variance:  σ² = [2v₂²(v₁ + v₂ - 2)]/[v₁(v₂ - 2)²(v₂ - 4)] 
           for v₂ > 4.

Example:

Suppose you randomly select 7 marbles from company 1's 
production line and 12 marbles from company 2's production 
line and measure their diameters. Assume you are given:

s₁ = 1.0 and s₂ = 1.1

F = s₁²/s₂² = 1/1.21 = 0.83

H₀:  σ₁ = σ₂
H₁:  σ₁ ≠ σ₂

From tables, F_0.05 for v₁ = n₁ - 1 = 6 and v₂ = n₂ - 1 = 11 is 
equal to 3.0946.  Since 0.83 < 3.0946 the result is not 
significant at the 0.05 and there is insufficient reason to 
reject H₀.