What is the physical meaning of ensemble average?

Question

I understand, I think, abstractly what an ensemble average is. However, I've always been confused by the physical meaning of it. More specifically, is it physically possible to obtain an ensemble average in reality? Since a system can only have one chosen state in an instance in time, one would never know what the ensemble average is because it is not possible for a system to go through all possible states at once.

Answer 1

It's a subtle notion because, very often an ensemble, as you note, is only a hypothetical population of potential thermodynamic (or in general stochastic) systems with the same statistical parameters / thermodynamic macrostate as the one under consideration. They don't all exist at once in physical reality. So often there is no direct, frequentist, physical interpretation of the ensemble average nor of any statistical estimator's "expected" value for the ensemble.

As a mathematical definition / thought experiment, one is to imagine the set of potential systems and one is to calculate statistics and probabilities measure-theoretically from that set. Since the set has no physical reality, often we must exploit obvious symmetries or make explicit postulates to succeed in such a calculation.

Often one also makes explicit postulates that certain different statistics are in fact equal to the ensemble version of a statistic notwithstanding the difference. In signal theory, for example, one makes the ergodic hypothesis that ensemble power spectral density (for example - the notion holds good for any statistic) for the full ensemble of signals of a certain kind is the same as the power spectral density estimated by time averaging of PSD estimators applied to the one signal. That is, roughly, that each instance of a signal, given enough time, will show all the behaviors belonging to the ensemble. Once one makes this hypothesis, the Wiener-Khinchin theorem tells us that we can calculate power spectral densities as Fourier transforms of time autocorrelation functions calculated for any signal instance.

In thermodynamics, the analogous ergodic hypothesis is that all potential system microstates of a system are equally likely and therefore, by observing one instance of that system for long enough, the random fluctuations of the system's microstate will carry the actual microstate through a statistically unbiased and representative sample of all the potential microstates.

In summary, ensemble statistics do not have a physical, frequentist interpretation because they are defined on a set of potential systems, and one makes explicit postulates to substitute other statistics for them on the grounds that (1) such substitution is plausible and (2) the results from these postulates seem to agree with experiment.

Answer 2

I think WetSavannaAnimal aka Rod Vance's answer already covers the precise physical meaning of the ensemble average. I will just briefly try to give an intuitive physical meaning.

Imagine you have a fluctuating source. A good example is the electric field¹ from real light-sources (lightbulbs, stars etc.). It may look something like this:

You could describe the property of such a signal as an average over time. But this is hard, since in most cases it is impossible to predict what the real time shape of the signal will look like.

Instead ⁽²⁾ we can often equivalently obtain may properties of the signal by doing an average over the statistical ensemble. To address the question

Since a system can only have one chosen state in an instance in time, one would never know what the ensemble average is because it is not possible for a system to go through all possible states at once.

An ensemble is hence not defined by the signal at one time, instead it characterises the average properties (such as mean power etc.) of the signal. Hence ensembles are designed such that we do not have to do nasty time averages⁽³⁾.

---

^{(1) working in a classical picture for simplicity}

^{(2) for when it is possible to apply this, see lemon's comment and WetSavannaAnimal aka Rod Vance's answer}

^{(3) how to compute the ensemble averages is then a whole different question, but there are many ensembles who's properties are known. First and foremost thermal ensembles}

Answer 3

To perform an ensemble average $\langle A\rangle$ of the variable $A$ in real life, you should take a large (ideally infinite) number of copies of your system, all in the same macrostate, measure $A$ in all of them and then do the average.

This is of course impractical, so an assumption must be made, called the ergodic hypotesis. The key is that a system in a certain macrostate is exploring an enormous number of different microstates every second, and the timescale of a measurement is usually of the order of seconds. Therefore every measurement is actually a time average $\bar A$ over the trajectory of your system in phase space. So if we suppose that every microstate corresponding to a given macrostate is equally probable, we will have, if the measurement lasts long enough,

$$\langle A \rangle \simeq \bar A$$

This relation becomes theoretically exact if the time of the measurement is infinite, but from a practical point of view this means that the duration of the measurement must be much larger than the typical timescales of our system (something of the order of picoseconds for atomic or molecular systems).

Notice however that there are some systems in which the ergodic hypotesis is not true, one famous example being glasses.

Answer 4

When a system can exist in a number of different possible states (called micro-states), an ensemble average is an average over all of those states, weighted by the likelihood of the system being found in each micro-state. It corresponds to the expectation value in probability theory, and is often a theoretical rather than practical concept.

In contrast, a time average follows the system step-wise through its evolution from one micro-state to another, and calculates the average over many steps such that all possible micro-states have been sampled in a representative manner. The time average could be found by experiment or simulation, or possibly by integration along a path of time-evolution.

For example, suppose we have a 6-sided die with faces 1,1,1,2,3,3. There are 3 possible micro-states (1,2,3) with probabilities $\frac36,\frac16,\frac26$. So the ensemble average (expectation value) of a roll is $\frac36*1+\frac16*2+\frac26*3=\frac{11}{6}=1\frac56$. The time-average of the value of a roll of this die would be found by rolling the die a large number of times. Ideally for a fair die this average will also equal $\frac{11}{6}$.