• Contact
• About

  Search by keyword:  

Last modified: January 26, 2018

Pearson Correlation
-------------------

Linear regression is similar to correlation in that the purpose 
is to measure to what extent there is a linear relationship 
between two variables. The major difference between the two 
is that correlation makes no distinction between independent 
and dependent variables while linear regression does. 

ρ = population correlation coefficient.

ρ = (1/N)Σ[(xi - μx)/σx][(yi - μy)/σy]

Pearson r = sample correlation coefficient.
                       _            _
r = (1/(n - 1))Σ[(xi - x)/sx][(yi - y)/sy]

                  SSxy
  = -------------------------------------
    (n - 1)√[(SSxx/(n - 1)]√[SSyy/(n - 1)]

       SSxy
  = ----------
    √(SSxxSSyy)

Alternatively,

r = √(SSM/Total SS) 

Where:
             _       _
SSxy = Σ(xi - x)(yi - y) 
             _       
SSxx = Σ(xi - x)2 
             _
SSyy = Σ(yi - y)2

Total SS = Model SS + Error SS

         = SSM + SSE
       _            _
Σ(yi - y)2 = Σ(yp - y)2 + Σ(yi - yp)2
                    _
           = Σ(yp - y)2 + Σ(yi - (βxi + α))2 

Coefficient of Determination:

r2  = Proportion of the variance of the dependent variable (y) 
      that is explained by the independent variable (x).

  = (SSyy - SSE)/SSyy

Hypotheses:

If there is a significant correlation between the variables, 
r will not equal zero.

H0: ρ = 0 
H1: ρ ≠ 0

Test statistic:  t test = r√[(n - 2)/(1 - r2)]

                        = β/sβ

                        =  β/(s/√SSxx)
                           
                           The test has n - 2 df.

General rule of thumb:

ρ:  0.10 - 0.29 small correlation
    0.30 - 0.49 medium correlation
    0.50 - 1.0 strong correlation

A simple example:
                 _         _         _       _
   xi  yi  (xi - x)2 (yi - y)2 (xi - x)(yi - y)
   -  ---  -------  -------   --------------
   1  1.9   2.25     0.8556       1.3875
   2  2.3   0.25     0.2756       0.2625
   3  2.8   0.25     0.0006      -0.0125
   4  4.3   2.25     2.1756       2.2125
   -  ---   ----     ------       ------
 Σ          5.00     3.3075       3.8500
_
x = 2.5000
_
y = 2.8250
SSxx = 5.0000
SSxy = 3.8500
SSyy = 3.3075
β = SSxy/SSxy = 3.8500/5.0000 = 0.7700 
SSE = SSyy - βSSxy = 3.3075 - 0.7700*3.8500 = 0.3430
    = Σ(yi - yp)2 = 0.3430
                               _
    yi    yp  (yi - yp)2  (yp - y)2
   ---  ----  ------     ------
   1.9  1.67  0.0529     1.3340
   2.3  2.44  0.0196     0.1482 
   2.8  3.21  0.1681     0.1482
   4.3  3.98  0.1024     1.3340
              ------     ------
 Σ            0.3430     2.9645

Total SS = SSM + SSE
       _            _
Σ(yi - y)2 = Σ(yp - y)2 + Σ(yi - yp)2

3.3075 = 2.9645 + 0.3430

Coefficient of Correlation:

r = √(SSM/Total SS)

  = √(2.9645/3.3075)

  = 0.9466

  = SSxy/√(SSxxSSyy)

  = 3.8500/√(5.0000*3.3075)

  = 0.9466

Coefficient of Determination:

r2 = (SSyy - SSE)/SSyy

  = (3.3075 - 0.3430)/3.3075

  = 0.8963

s2 = SSE/(n - 2) = 0.3430/2 = 0.1715 ∴ s = 0.4141

t = β/(s/√SSxx) = 0.77/(0.4141/√5.0000) = 4.158

t test = r√[(n - 2)/(1 - r2)]

       = 0.9466√[2/(1 - 0.8962)]

       = 4.155

t0.05/2 for 4 - 2 = 2 df is 4.303 (2-tailed). Therefore, 
reject H0.


SPSS:  Analyze>Correlate>Bivariate

Multiple Correlation
--------------------

Because we now have multiple variables, we have more than one 
correlation value to consider => partial correlation coefficient.  
PCC is the measure of the correlation between the dependent 
variable and one independent variable holding constant the 
influence of all other independent variables.  

Correlation matrix:

    x1  x2  x3  x4
    x2  1   ?   ? 
    x3  ?   1   ?
    x4  ?   ?   1

Test statistic:  One-sample t test = r√[(n - 2)/(1 - r2)]

SPSS:  Analyze>Correlate>Bivariate

Newsletter Sign-up:

Copyright © 2023 Redshift Academy

website built by me using the Codeigniter MVC framework