Correspondence between eigenvalues and eigenvectors in ellipsoids

Question

think of an ellipsoid in the n-dimensional space defined by $$(x-\mu)'A(x-\mu)=1.$$

I was calculating the volumes of n-dimensional ellipsoids like the one from above for a while, which is straightforward once the eigenvalues of matrix $A$ are retrieved. The volume $V$ is then given by (using the log scale so that it will not create numerical problems for very large $n$) $$ V_{0} = log(1),\\ V_{1} = log(2), $$ and then for any $i>1$ $$ V_{i} = log(\frac{2\pi}{i-1}) + V_{i-2},$$ and finally $$V = V_{n} + \sum\limits_{i=1}^{n} log(\lambda_{i}^{-\frac{1}{2}}),$$ where each $\lambda_{i}$ refers to one of the $n$ eigenvalues of $A$.

Now we know that each eigenvector of $A$ is the orientation of an axis of the ellipsoid and each $\lambda^{-\frac{1}{2}}$ refers to the half length of one of these axes.

Nevertheless, most (all that I know) mathematical routines that calculate the eigenvalues of $A$ will give the result in a meaningless order. Thus, I will know all half length of the axes of the ellipsoid, but I do not know which eigenvalue corresponds to which eigenvector. I can find this out (I guess) when testing for each eigenvector $v_{k}$ if $$ Av_{k}=\lambda_{i}v_{k}$$ and the $\lambda_{i}$ for which the above holds, will then be the eigenvalue corresponding to $v_{k}$. Nevertheless, my software (R) return eigenvectors normalized to length 1, so the above identity never exists.

My question would be if someone could help me to find out how to sort the eigenvalues anyway so that the first eigenvalue corresponds to the first eigenvector and so on...

Every help is highly appreciated, J

PS: my post is somewhat related to this post and this post, but finally could not figure out how to answer my question with these posts.

Answer 1

They are associated in R

I'll concentrate on the eigen function from R, as you mention it in a comment. Quoting from its manual:

If ‘r <- eigen(A)’, and ‘V <- r$vectors; lam <- r$values’, then
                     A = V Lmbd V^(-1)                         


(up to numerical fuzz), where Lmbd =‘diag(lam)’.

This implies that the columns of V will match the values in lam in order. The first column of V will always be the eigenvector for the first (i.e. maximal) eigenvalue. lam[i] will always correspond to V[,i].

Eigenvector scale and normalization

You stated:

Nevertheless, my software (R) return eigenvectors normalized to length 1, so the above identity never exists.

But this is not true. Quoting again from the manual:

Recall that the eigenvectors are only defined up to a constant: even when the length is specified they are still only defined up to a scalar of modulus one (the sign for real matrices).

You can verify this easily: if some $v$ satisfies $Av=\lambda v$, then a multiple $w=\mu v$ for some $\mu\neq0$ also satisfies $Aw=\mu Av=\mu\lambda v=\lambda w$ so it, too, will be an eigenvector. This is one of the reasons why it might make more sense to speak about eigenspaces, which are the linear spaces spanned by an eigenvector. The eigenvector isn't uniquely defined by the eigenvalue, but the eigenspace in general is. Choosing a vector of length one is just a choice which is useful in many situations.

Further concerns

Your comment also states that, if A is a covariance matrix, then “the first eigenvector corresponds to the first variable”. I am not sure what you mean by an eigenvector corresponding to a given variable, since these vectors will always have entries for all of the variables. Using some randomized data:

> set.seed(1)
> somedata<-data.frame(x=runif(50), y=runif(50), z=runif(50))
> (A <- cov(somedata))
             x            y            z
x  0.074114067  0.003904344 -0.001744775
y  0.003904344  0.070106650 -0.004266336
z -0.001744775 -0.004266336  0.077143995
> eigen(A)
$values
[1] 0.08086722 0.07362015 0.06687733

$vectors
           [,1]      [,2]       [,3]
[1,] -0.4643667 0.7911076  0.3981360
[2,] -0.4668754 0.1633423 -0.8691068
[3,]  0.7525894 0.5894642 -0.2934980

I see no indication in that eigenvector matrix which supports the claim that the first column were in some way associated with the variable x. So I cannot support the assumption you stated in your comment.

The statement from the manual can indded be verified on this data:

> r <- eigen(A)
> V <- r$vectors; lam <- r$values
> Lmbd <- diag(lam)
> round(V %*% Lmbd %*% t(V) - A, digits=16)
  x y z
x 0 0 0
y 0 0 0
z 0 0 0

If you don't trust that formulation, you may prefer a more explicit eigenvector equation like this:

> round(A %*% V[,1] - lam[1]*V[,1], digits=16)
  [,1]
x    0
y    0
z    0

As a link between these views, you can vectorize the latter for all eigenvectors to obtain a reformulation of the former:

> round(A %*% V - V %*% Lmbd, digits=16)
  [,1] [,2] [,3]
x    0    0    0
y    0    0    0
z    0    0    0

The left hand side of the difference is the matrix A applied to all eigenvectors, whereas the right side is the eigenvectors scaled by their eigenvalues. The differences are zero, so the two are equal. This verifies that the order matches as expected.