Physics · Molecular physics and thermodynamics · Grade 9

Entropy and the arrow of time

Why don't mixed gases separate by themselves — and what if we ran time backwards?

The situation

If you put a spoonful of sugar into tea, it dissolves and spreads through the whole cup — but nobody has ever seen the sugar spontaneously gather in one place again. If perfume is sprayed in a corner of a room, the smell spreads through the whole room. Yet Newtonian mechanics is symmetric in time: every collision can also be “played” in reverse. So why do processes go only one way? Take away the barrier that separates two gases, and then try turning time backwards.

Topic

Statistical physics and entropy. Mixing of gases: why the reverse never happens by itself

Key ideas

Microstate and macrostate
S = k ln W
Loschmidt's paradox

Formulas

S = k \ln W

Boltzmann's formula: the entropy of a macrostate is determined by the number of microstates that realise it — the statistical weight $W$ . $k = 1.38 \cdot 10^{- 23}$ J/K.

W = C_{N}^{n} = \frac{N!}{n! (N - n)!}, P = 2^{- N}

The number of ways in which $n$ of $N$ particles are in the left half (greatest at $n = N / 2$ ), and the probability that all the particles by chance gather in their “own” half: for $N = 400$ , $\approx 10^{- 120}$ .

S = - k \sum_{i} p_{i} \ln p_{i}

Coarse-grained (by cells) entropy: $p_{i}$ is the probability that a particle is in cell $i$ (and is of that type). $- S / k$ is Boltzmann's $H$ function.

Δ S = N k \ln 2

The increase in entropy when two gases mix, or when a gas expands into a vacuum and its volume increases by a factor of 2.

d S \geq \frac{δ Q}{T}, F = U - T S, A_{m a x} = - Δ F

The Clausius inequality (equality only for a reversible process) and the free energy: the maximum work that can be obtained at constant temperature.

Theory

Microstate, macrostate and ensemble

A macrostate is what we measure: how many blue particles there are in the left half. A microstate is the exact position and velocity of every particle. Many microstates give one macrostate; their number $W = C_{N}^{n}$ is the statistical weight (thermodynamic probability) of the macrostate. The experiment shows three ways of studying matter: dynamic (the computer solves the equation of motion of every disc), statistical (probabilities and $W$ ) and thermodynamic (only a macroscopic quantity — entropy). In a closed tray the energy does not change (E/E₀ in the panel); all possible microstates of such an isolated system are equally probable — the microcanonical distribution, so $P (n) = W (n) / 2^{N}$ . The curve on the graph is the distribution over the ensemble, and the bars are a histogram collected over time by watching one tray for a long time: the two almost coincide (because of the finite size of the discs the histogram is slightly narrower). The particles are statistically independent, so the probabilities are multiplied: $P = {(1 / 2)}^{N}$ . In quantum mechanics the ensemble is described by the density matrix.

Entropy increase and the second law

The box is divided into $6 \times 3$ cells. The more evenly the particles are spread over the cells and types, the larger $S = - k \sum p \ln p$ . Once the barrier is removed, $S$ tends to the maximum — $Δ S = N k \ln 2$ — and fluctuates slightly around it; the fluctuations shrink as $1 / \sqrt{N}$ . This is a manifestation of Boltzmann's H-theorem: as a result of collisions $H = - S / k$ decreases — the law of increase of entropy. In an isolated system equilibrium is the maximum of entropy; deviations from it are less probable, so the equilibrium is stable (stability conditions: $C_{V} > 0$ , ${(\partial p / \partial V)}_{T} < 0$ ). The classical statements of the second law: Clausius — heat does not pass spontaneously from a cold body to a hot one; Thomson (Kelvin) — there is no cyclic machine that converts heat taken from a single heat source completely into work. In Boltzmann's interpretation the second law is a statistical law: for $N = 6$ the gases may spontaneously separate again, but for $N = 400$ this would take longer than the age of the universe. Radiation also has entropy — in the “Thermal radiation and Planck” experiment.

Reversible and irreversible processes

A reversible (quasi-static) process goes infinitely slowly through a sequence of equilibrium states and can also run in the opposite direction: for example, slowly mixing gases with semipermeable pistons. Removing the barrier suddenly is irreversible: the intermediate states are not in equilibrium. For any process the Clausius inequality holds, $d S \geq δ Q / T$ , with equality only for a reversible process. From it comes the fundamental equation of thermodynamics: for a reversible process $T d S = d U + p d V$ , for an irreversible one $T d S > d U + p d V$ . In “Gas → vacuum” mode $Q = 0$ but $Δ S = N k \ln 2 > 0$ — the inequality is strict. The cells show local equilibrium: each cell is almost in equilibrium on its own, while the particles flow between neighbouring cells — for each type there is a balance equation: the change in the number of particles in a cell = incoming flow − outgoing flow (the particles are conserved). The flows produce entropy: the panel shows $d S / d t \geq 0$ , and at equilibrium it falls to zero.

Loschmidt's paradox and the model

If we exactly reverse all the velocities, then by Newton's laws the particles retrace their paths and the gases separate again. In this simulation the calculation is done in integers, so “Time backwards” returns every particle exactly to its initial place. But if an error of $10^{- 12}$ is introduced into the particle positions, each collision magnifies it several times over (chaos): after a few dozen collisions the “memory” is lost. So the microscopic laws are reversible, but macroscopic processes are irreversible — irreversibility follows from probabilities and initial conditions. For a dilute gas this is described by Boltzmann's kinetic equation (in the “Transport phenomena” experiment). Model: two-dimensional discs; collisions are very hard elastic repulsion, and the kinetic energy is conserved to ≈ 2 % (E/E₀ in the panel). The integrator is the integer “leapfrog” (the Levesque–Verlet method). Because $N$ is finite, the maximum of the measured $Δ S$ is slightly below $N k \ln 2$ (≈ 96 % for $N = 400$ ).

Free energy, the third law and negative temperature

For systems that exchange heat with their surroundings, thermodynamic potentials are convenient: the internal energy $U$ , the enthalpy (heat function) $H = U + p V$ , the free energy $F = U - T S$ and the Gibbs potential $G = H - T S$ . From their differentials come the Maxwell relations: for example, from $d F = - S d T - p d V$ we get ${(\partial S / \partial V)}_{T} = {(\partial p / \partial T)}_{V}$ ; the Gibbs–Helmholtz equation: $U = F - T {(\partial F / \partial T)}_{V}$ . At constant $T$ a system tends to the minimum of $F$ , and at constant $T$ and $p$ to the minimum of $G$ (phase equilibrium — in the “Phase transitions” experiment). When ideal gases mix, $U$ does not change and $Δ F = - T Δ S$ : if the mixing were done reversibly, work $T Δ S$ could be obtained — in the panel this is the “maximum work lost”. The third law (Nernst's theorem): as $T \to 0$ the entropy tends to zero ( $W = 1$ ), so the heat capacities also tend to zero and absolute zero cannot be reached in a finite number of steps. A two-level system: if $n$ of $N$ spins are in the upper level ( $ε$ ), then $W = C_{N}^{n}$ — the graph below is exactly this. $1 / T = \partial S / \partial E = (k / ε) \ln \frac{N - n}{n}$ : if $n > N / 2$ (a population inversion of levels, as in a laser), $T < 0$ — a negative absolute temperature; such a system is “hotter” than any body with a positive temperature: heat always flows out of it.

Maxwell's demon

In 1867 Maxwell proposed this thought experiment: a “demon” operates a door in the barrier — it lets a blue particle pass only to the left and an orange one only to the right. No work is done, yet the gases separate and the entropy decreases — as if the second law were violated. See this in “Maxwell's demon” mode (in the model the door is the middle third of the barrier): $Δ S < 0$ . The solution of the puzzle is information. The demon watches each approaching particle, makes a decision and writes it into its memory — each decision is 1 bit. Memory is not unlimited: erasing it releases at least $k T \ln 2$ of heat (Landauer's principle, 1961). In the panel the number of demon decisions gives far more than the fall in gas entropy: $n_{b i t} k \ln 2 > | Δ S |$ — together with the demon the total entropy does not decrease. Entropy is often called a measure of disorder; more precisely, it is the number of microstates corresponding to the macrostate — a measure of the missing information about the system.

Did you know?

Ludwig Boltzmann's entropy formula $S = k \log W$ is carved on his gravestone in Vienna.
The objection about reversing time was raised in 1876 by the Austrian physicist Josef Loschmidt — that is why it is called “Loschmidt's paradox”.

Inside the experiment

The experiment starts from a real-life situation: you predict the result, change the controls, run the experiment and draw a conclusion. Inside are missions, theory, a quiz and a guided tour in which the camera moves and explains the key points.

More experiments in this section

Calorimeter: heat balance — Grade 7
Tandoor and sandal: heat — Grade 7
Gas under a piston — Grade 9
Humidity and dew — Grade 9
Heat engine: the Otto cycle — Grade 9
Brownian motion and diffusion — Grade 9
Surface tension and capillarity — Grade 9
Crystalline and amorphous solids — Grade 9
Phase transitions — Grade 10
Thermal expansion — Grade 10