13

I'm trying to figure out if there is a direct relationship between these concepts. Strictly from the definitions, they appear to be different concepts in general. The more I think about it, however, the more I think they are very similar.

Let $X,Y$ be WSS random vectors. The covariance, $C_{XY}$, is given by $$C_{XY}=E\left[(X-\mu_x)(Y-\mu_y)^H\right]$$ where $H$ stands for the Hermitian of the vector.

Let $Z$ be a WSS random vector. The autocorrelation function, $R_{XX}$, is given by $$R_{ZZ}(\tau)=E\left[\left(Z(n)-\mu_z\right)\left(Z(n+\tau)-\mu_z\right)^H\right]$$

Edit Note There is a correction to this definition as applied to signal-processing, see Matt's Answer below.

The covariance does not involve a concept of time, it assumes each element of the random vector is a different realization of some random generator. The autocorrelation assumes a random vector is the time evolution of some initial random generator. Yet in the end, they are both the same mathematical entity, a sequence of numbers. If you let $X=Y=Z$, then it appears $$C_{XY}=R_{ZZ}$$ Is there something more subtle that I am missing?

ijuneja
  • 105
  • 5
rbell
  • 231
  • 1
  • 2
  • 4
  • The definition of AutoCorrelation $R_{ZZ}(\tau)$ is incorrectly stated in the question as pointed out by Matt – ijuneja Oct 20 '18 at 06:07

2 Answers2

11

According to your definition of autocorrelation, the autocorrelation is simply the covariance of the two random variables $Z(n)$ and $Z(n+\tau)$. This function is also called autocovariance.

As an aside, in signal processing, the autocorrelation is usually defined as

$$R_{XX}(t_1,t_2)=E\{X(t_1)X^*(t_2)\}$$

i.e., without subtracting the mean. The autocovariance is given by

$$C_{XX}(t_1,t_2)=E\{[X(t_1)-\mu_X(t_1)][X^*(t_2)-\mu^*_X(t_2)]\}$$

These two functions are related by

$$C_{XX}(t_1,t_2)=R_{XX}(t_1,t_2)-\mu_X(t_1)\mu^*_X(t_2)$$

Matt L.
  • 89,963
  • 9
  • 79
  • 179
  • If you look at $\tau$ as a variable, then the autocorrelation becomes a function of that "time gap" which can yield very interesting information about the data set. Look at the relation between autocorrelation, discrete Fourier transforms and the Wiener–Khinchin theorem. – PhilMacKay Feb 28 '17 at 16:36
  • @PhilMacKay: Sure, but that only works for WSS processes. I gave the definitions for the general case, where processes are not necessarily stationary. – Matt L. Feb 28 '17 at 16:43
  • Yes indeed non-stationary processes can be annoying for data analysis, which is why I always try to de-trend a data before using my beloved statistical tools! It's not always possible, though... – PhilMacKay Feb 28 '17 at 20:05
0

Notice how your definition of Autocorrelation includes an additional term $\tau$, which specifies an offset from the two sequences of number $Z(n)$ and $Z(n+\tau)$. In fact, the notation suggest that $R_{ZZ}(\tau)$ is a continuous function defined for any $\tau \in \mathbb{R}^+$, while $C_{XY}$ is a scalar.

As you mentioned, if you let $X=Y=Z$, then you are implying that $\tau = 0$, which is one special case of $R_{ZZ}(\tau)$.

In my personal experience (astrophysics, various sensor processing), the covariance was used as a coefficient to check the similarity of two datasets, while the autocorrelation was used to characterize the correlation distance, that is, how quickly a data evolves to become another data entirely.

PhilMacKay
  • 519
  • 2
  • 9