4

Hi All: I'm trying to better understand the connection between variance of a time series and the integral of the spectral density over all frequencies. Rather than going through all of the relations, if one looks at the link below,

https://en.wikipedia.org/wiki/Spectral_density, the relations are shown there in the power spectral density section.

To summarize what is said there:

The power, $P$, is defined as the limit as $P=\lim_{T \to \infty} \frac{1}{2T} \int_{-T}^{T} |x(t)|^2$.

$S_{xx}(\omega)$ is defined as the limit as $S_{xx}(\omega)=\lim_{T \to \infty} E|\hat{x}_T(\omega)|^2$. So, I think of it as the expected instantaneous power at frequency $\omega$. I hope that's the way to think of it.

$\hat{x}_{T}(\omega)$ is defined as $\frac{1}{\sqrt T} \int_0^{T} x(t) e^{-i\omega t} dt$. So the fourier transform of $x_t$ divided by $\sqrt T$.

Finally, $\gamma_{0}$ is defined as the power contained in the total frequency band so $\int_{\omega} S_{xx} (\omega)$. My confusion is the following:

In the time domain-econometrics world, $\gamma_{0}$ is defined as $\mathrm{Var}(X_{t}) = E(X_{t} - \mu)^2$ where $\mu$ is the mean of $X_{t}$.

So, my confusion stems from the fact that when the variance is defined as power, there is no mean subtracted from $X_{t}$. Does that mean that $X_{t}$ has already been de-meaned when it's written as $X_t$. If not, then how can $\gamma_{0}$ be viewed as the variance when statistical variance is the (expected value of the deviation of a random variable from it's mean) squared yet the view of variance in the DSP world is a weighted (by frequency) combination of $E(X_{t}^2)$ over all frequencies. My only conclusion is that the mean is already being subtracted out or is assumed to be zero ? I basically don't understand how variance is defined in the DSP framework. Is it viewed as expected power rather than expected variability ? Thanks.

Matt L.
  • 89,963
  • 9
  • 79
  • 179
mark leeds
  • 1,117
  • 1
  • 7
  • 14

1 Answers1

5

Variance is never defined as power. For a wide-sense stationary random process $X(t)$ with zero mean

$$\mu_X=E\{X(t)\}=0\tag{1}$$

the variance of $X(t)$ equals its power.

The autocorrelation of $X(t)$ is defined by

$$R_X(\tau)=E\{X^*(t)X(t+\tau)\}\tag{2}$$

The power of $X(t)$ is

$$P_X=E\{|X(t)|^2\}=R_X(0)\tag{3}$$

The variance of $X(t)$ is

$$\sigma^2_X=E\{|X(t)-\mu_X|^2\}=E\{|X(t)|^2\}-|\mu_X|^2=R_X(0)-|\mu_X|^2\tag{4}$$

Clearly, if $\mu_X=0$, we have $\sigma^2_X=R_X(0)=P_X$.

Note that the autocovariance of $X(t)$ is defined by

$$C_X(\tau)=E\{[X(t)-\mu_X]^*[X(t+\tau)-\mu_X]\}=R_X(\tau)-|\mu_X|^2\tag{5}$$

Consequently,

$$\sigma^2_X=C_X(0)\tag{6}$$

But since the power spectral density $S_X(\omega)$ is the Fourier transform of $R_X(\tau)$, the integral over $S_X(\omega)$ equals the power of $X(t)$ and not its variance, unless $\mu_X=0$:

$$P_X=R_X(0)=\frac{1}{2\pi}\int_{-\infty}^{\infty}S_X(\omega)d\omega\tag{7}$$

Also take a look at this related answer.

For the difference between the definition of autocorrelation in statistics and in signal processing look here.

Matt L.
  • 89,963
  • 9
  • 79
  • 179
  • Hi Matt: The issue must be notational then because, in statistical-time series and time series econometrics $R_{x}(0)$ is the autocovariance at lag zero which is the variance. ( no assumptions about zero mean ). Clearly, in your explanation ( which was quite nice by the way ), $R_{x}(0)$ is only the variance when $u_x = 0$. But the question kind of still remains then in that how does DSP deal with variance when the mean isn't zero ? Thanks for nice explanation. – mark leeds Jan 14 '18 at 18:30
  • Matt: The confusion basically stems from the fact that, in the non DSP world, $R_{x}({\tau})$ is defined as $E[(X_t+\tau - \mu)(X_t - \mu)$ so this is what caused my confusion. But, based on what you said, you have to demean the data in order to compute the variance using DSP techniques which is not difficult. Thanks. – mark leeds Jan 14 '18 at 18:41
  • @markleeds: Yes, then it must be notation. Variance is the autocovariance at lag $\tau=0$, not the autocorrelation at lag $\tau=0$, unless the mean is zero. What do you mean with how DSP deals with variance in the non-zero mean case? Just as it should, namely by considering the mean and not equating power and variance. – Matt L. Jan 14 '18 at 18:42
  • @markleeds: In DSP we usually use $R_X(\tau)$ to denote the autocorrelation, not the autocovariance (which we don't use very often). In your world, apparently $R_X(\tau)$ is the autocovariance. No big deal if that's all. – Matt L. Jan 14 '18 at 18:45
  • Matt: I mean ( no pun intended ) that, if you want to calculate the variance of your time series and you want to stay in DSP world, then you need to demean the data and then calculate the power by using the spectral density of the de-meaned series. Is that correct ? – mark leeds Jan 14 '18 at 18:45
  • @markleeds: There are many ways. You don't need to use the PSD at all. You might as well just estimate the power and the mean in the time domain. But it is indeed the case that in many applications the random processes we consider (are assumed to) have zero mean. So then variance and power are indeed equal. – Matt L. Jan 14 '18 at 18:49
  • @markleeds: I added two sentences at the end of my answer. The last one is about different definitions of autocorrelation in statistics and signal processing. – Matt L. Jan 14 '18 at 18:55
  • So, in conclusion: the variance calculate from the normalized periodogram will be equal to the time domain variance? – Sapiens Oct 03 '20 at 18:33