Why isn't the entropy of gas infinite if there are infinite microstates available?

Question

How could it be that the entropy of gas is not infinite? If we use the formula $S=k\ln(\Omega)$ we get infinity because there are an infinite number of possible situations (because there are infinite possibilities of position and momentum for each molecule).

Answer 1

The problem you are thinking about is known as the question of thermodynamic coarse graining. This will hopefully give you a phrase that you can search on to find out more.

Sometimes possible states of ensemble members are obviously discrete, as they are in a collection of quantum harmonic oscillators. Much of quantum mechanics depends on the underlying Hilbert state space's being separable (i.e. has a countable dense subset) and for Hilbert spaces this is equivalent to the assertion that the vector basis itself is countable. Thus, even if if an observable such as momentum or position has a continuous spectrum (i.e. can give a continuous random variable as a measurement), the underlying state space is often discrete. In the OPs particular example, you can model the gas as a system of particles in a 3D box (hat tip to user knzhou for reminding me of this point), so that the state space of ensemble members is clearly discrete. As we raise the volume of our box, the density of states (discussed more in Chris's answer) increases in proportion with the box's spatial volume, and therefore so does the entropy. In the limit of a very large gas volume, the entropy per unit volume is a well defined, finite limit.

In cases where the state space is not obviously discrete, one must resort to either the use of coarse graining or relative entropy.

Coarse graining is the somewhat arbitrary partitioning of ensemble members' state space into discrete subsets, with states belonging to a given partition then being deemed to be the same. Thus a continuous state space is clumped into a discrete approximation. Many conclusions of statistical mechanics are insensitive to such clumping.

Relative entropy, in the information theoretic sense, is defined for a continuous random variable as an roughly the entropy change relative to some "standard" continuous random variable, such as one governed by a Gaussian distribution. We see the problem you are dealing with if we try naïvely to work out the Shannon entropy of a continuous random variable with probability distribution $p(x)$ as the limit of a discrete sum:

$$S \approx -\sum\limits_i p(x_i)\,\Delta x \,\log(p(x_i)\,\Delta x) = -\log(\Delta x)\,\sum\limits_i p(x_i)\,\Delta x - \sum\limits_i p(x_i)\,\Delta x \,\log(p(x_i))\tag{1}$$

The two sums in the rightmost expression converge OK but we are thwarted by the factor $\log(\Delta x)$, which of course diverges. However, if we take the difference between the entropy for our $p(x)$ and that of a "standard" distribution, our calculation gives:

$$\Delta S \approx -\log(\Delta x)\,\left(\sum\limits_i p(x_i)\,\Delta x-\sum\limits_i q(x_i)\,\Delta x\right) - \sum\limits_i \left(p(x_i)\,\log(p(x_i))-q(x_i)\,\log(q(x_i))\right)\,\Delta x\tag{2} $$

a quantity which does converge to $\int\left(p\log p - q\,\log q\right)\mathrm{d}x$. The usual relative entropy is not quite the same as this definition (see articles - the definition is modified to make the measure independent of reparameterization) but this is the the basic idea. Often the constants in the limit of (2) are dropped and one sees the quantity $-\int\,p\,\log p\,\mathrm{d}x$ defined as the unqualified (relative) entropy of the distribution $p(x)$.

Coarse graining, in this calculation would be simply choosing a constant $\Delta x$ in (1). (1) is then approximately the relative entropy $-\int \,p\,\log p\,\mathrm{d}x$ offset by the constant $-\log(\Delta x)$. Therefore, as long as:

1. We stick with a constant $\Delta x$ in a given discussion;
2. $\Delta x$ is small enough relative to the variations in the probability density so that $\sum\limits_i p(x_i)\,\Delta x \,\log(p(x_i))\approx \int p\,\log p\,\mathrm{d} p$;
3. Our calculations and physical predictions are to do only with differences between entropies (as is mostly the case)

then the approaches of coarse graining and relative entropies give identical physical predictions, independently of the exact $\Delta x$ chosen.

A good review of these ideas, with historical discussion, is to be found in:

Katinka Ridderbos, "The coarse-graining approach to statistical mechanics: how blissful is our ignorance?", Studies in History and Philosophy of Modern Physics, 33, pp65-77, 2002

Answer 2

In a continuous system, $w$ is taken to be an integral over the possible microstates of the system. This is typically referred to as the density of states, and is quite finite. Specifically it's something like $$w(E)=\mathcal N\int d^{3N}x~d^{3N}p~\delta(E-\epsilon(\vec p,\vec x))$$

where $\epsilon(\vec p,\vec x)$ is the energy of the system as a function of all the momenta and positions, $\delta$ is a delta function that fixes the energy of the system and $\mathcal N$ is some normalization.

This leaves some ambiguity (namely, in the normalization), but the ratio of $w$ between two states is well-defined, so $\Delta S=k\ln(\frac{w_f}{w_i})$ is well-defined, and the ambiguity is resolved by defining entropy to be zero at absolute zero, which gives the normalization to use.

Answer 3

I advice you to have a look at any Statistical Mechanics textbook. This point is usually addressed in the first chapters. You want to compute the micro-canonical entropy $$S(E)=k_B\ln\Omega(E)$$ where $\Omega(E)$ counts the number of micro-states with energy $E$. As you pointed out, this number is infinite if the coordinates are continuous. The standard approach is to discretize the phase space and divide it in cells. Each cell is assumed to correspond to a single micro-state. For $N$ particles in a 3D space, the dimension of the phase space is $6N$. The width of a cell is chosen to be $\Delta q$ in the $3N$ directions associated to coordinates and $\Delta p$ in the $3N$ directions associated to momenta. It is then assumed that $$\Delta q\Delta p=h_0$$ Now the number of states $\Omega(E)$ with energy $E$ is finite and you can compute the microcanonical entropy. This is a special case of coarse-graining mentioned in the previous answers.

The advantage of this approach is that you can easily estimate $\Omega(E)$, the number of cells of energy $E$, as the volume $Vol$ of the phase space corresponding to an energy $E$ divided by the volume $h_0^{3N}$ of a cell (I simplify a bit here: what should be computed first is actually the volume of the phase space corresponding to an energy lower than $E$). This estimate is accurate only if $h_0$ is chosen sufficiently small. See what happen to the entropy: since $\Omega(E)=Vol/h_0^{3N}$, entropy reads $$S(E)=k_B\ln Vol-3Nk_B\ln h_0$$ The parameter $h_0$ appears only as an additive constant. Since thermodynamic averages are given by derivatives of the entropy, they do not depend on $h_0$ (hopefully since the discretization of phase space is not physical) and you can give any value to $h_0$, as long it remains sufficiently small. Usually, no definite value is given to $h_0$.