• Contact Us
• Programming Blog
• Tutoring Services
• Bio

  Search by keyword:  

Last modified: November 2, 2021

Definition of Entropy
---------------------

Entropy is a measure of our ignorance regarding the exact state of a
system.  Consider the example of a gas in a bottle.  The smaller the
bottle the less uncertainty there is about where the molecules will be
and so the entropy will be lower.  On the other hand for a constant
sized bottle the entropy will increase if heat is applied.  This is because
there will now be a wider range of possible velocities for the molecules
in the gas.  Now, if the bottle is opened in outer space, the gas will escape 
an diffuse into the vacuum.  Now there are progressively more and more 
states that the gas can be in and so the entropy increases.  Consequently,
as the system evolves in time the system becomes more and more 
disordered and our ignorance regarding the exact state of the system 
increases.

Definition:

We know state of system is somewhere in region defined by the distribution Pi 

Pi
|                     
|                   .  .   
|               .          .
|             .              .
|          .                    .   
|     .                              .
-------------------------------------------- State, i
         

Then,

S = -ΣiPilnPi

This is the average value of lnPi

Consider a specific case where all the states have the same Pi:

Pi
|                                     
|            
|         ----------                  
|        |          |           
|        |          |        
--------------------------- State, i
          <-- M -->
   
S = -ΣiPilnPi
 
S = -Σi(1/M)ln(1/M)

S = -M(1/M)ln(1/M)

S = -ln(1/M)

S = -lnM

Thus, in this case, S is just the number of states in the system.

Consider N coins:

N = 2n

S = lnN = nln(2)

Entropy
--------

The following discussion refers to distinguishable particles.   Two particles 
can be considered to be distinguishable if their separation is large compared 
to their DeBroglie wavelength. 

For example, the condition of distinguishability is met by molecules in an 
ideal gas under ordinary conditions.  On the other hand, two electrons 
in the first shell of an atom are inherently indistinguishable because of the 
large overlap of their wavefunctions. 

Consider N particles.  Let ni = number of particles in the ith energy state, Ei

Energy
|        
+
+
+<- ith state with Energy Ei
+
+
+
+
|

The number of possible ways to fit the particles in the number of 
avalable states is called the multiplicity function.

The multiplicity function for the whole system, W, is the product of 
the multiplicity functions for each energy Ei:

W = Πi(N!/ni)  ... 1.

Consider the following example of how 6 particles can be distributed amongst 
10 energy states.



For the 3 states shown above the numbers of ways each state can be
filled with 6 particles is given by:

6!/5!1! = 6

6!/3!2!1! = 60

6!/2!2!1!1! = 180

We can simplify 1. above by taking the log of both sides,

lnW = lnN! - Σiln(ni!)

Now apply Stirling's approximation i.e. lngA! = AlnA - A

So

lnW = NlnN - N - Σiniln(ni) + Σini

Now,

Pi = ni/N = probability that a given particle is in state i

Therefore,
 
lnW = NlnN - N - ΣiNPilnNPi + N

    = NlnN - (ΣiNPilnN + ΣiNPilnPi)

    = NlnN - (NlnN + NΣiPilnPi)

    = -NΣiPilnPi 

    = NS

where 

S = ΣiPilnPi

This is the definition of ENTROPY.

Boltzmann Distribution
------------------------

From the discussion of entropy it was shown that:

lnW = NS

    = NΣiPilnPi where W is the mutiplicity function.

Now maximize lnW by maximizing S.  This can be achieved by using 
the method of Lagrange multipliers under the constraints ΣPi = 1 
and E = ΣiPiEi (the average energy).  Thus,
      
lnW = -NΣiPilnPi - αΣiPi - βΣiEiPi

Now, for maximization, the derivative of each of the i's must 
equal 0.  Thus we can ignore the Σ.

∂lnW/∂Pi = -lnPi - 1 - α - βEi = 0 

∴ lnPi = -(1 + α) - βEi

Or 

Pi = exp(-(1 + α))exp(-βEi)

Write as,

Pi = (1/Z)exp(-βEi)  

This is the BOLTZMANN DISTRIBUTION.

The Partition Function
-----------------------

From the discussion of the Boltzmann distribution it was shown that:

Pi = (1/Z)exp(-βEi)

Now impose the above constraints ΣPi = 1 and E = ΣiPiEi  Thus,

ΣPi = 1:

(1/Z)Σie-βEi = 1 

∴ Z = Σie-βEi

This is the PARTITION FUNCTION

E = ΣiPiEi:

Take the derivative of Z w.r.t. β

∂Z/∂β = -ΣiEiexp(-βEi)

Divide both sides by -Z:

(-1/Z)∂Z/∂β = ΣiEiexp(-βEi)/Z

The RHS contains the Boltzmann distribution Pi = (1/Z)exp(-βEi) 

            =  ΣiPiEi

            = E

∴ E = (-1/Z)∂Z/∂β 

This can be written as:

E = -∂lnZ/∂β   

Entropy Revisited
-----------------

S = -ΣiPilnPi and Pi = (1/Z)exp(-βEi)

Therefore S can be wriiten as:

S  = -Σi(1/Z)e-βEilnPi  

Now, 

lnPi = (-βEi - lnZ).  Therefore,

S = -Σi(1/Z)e-βEi(-βPi - lnZ)

  = Σi(1/Z)e-βEi(βEi + lnZ)

  = βΣi(1/Z)e-βEiEi  + lnZ(1/Z)Σie-βEi

  = βΣiPiEi + lnZ(1/Z)z

  = βE + lnZ 

Now,

dS = βdE + Edβ + (∂lnZ/∂β)dβ

but ∂lnZ/∂β = -E so

dS = βdE 

or,

β = dS/dE

Helmholtz Free Energy
---------------------

Temperature is defined as:

T = (1/kB)dE/dS

The temperature, T, is the change in E that causes a change 
in the entropy by 1J/K.

Alternatively,

dS/dE = 1/KBT

By comparison,

β = 1/KBT

Now from before S = βE + lnZ.  Therefore,

S = (1/KBT)E + lnZ

Or

E - KBTS = A = -KBTlnZ 

This is the HELMHOLTZ FREE ENERGY.

Summarizing:



Pi = (1/Z)exp(-βEi) 

Z = Σie-βEi 

E  = -∂lnZ/∂β 

KBT = 1/β

S = βE + lnZ

A = -KBTlnZ

Newsletter Sign-up:

Copyright © 2020 Redshift Academy

website built by Redshift Academy using the Codeigniter MVC framework