7

Prove you can choose orthonormal bases $(e_1,...,e_k)$ and $(f_1,...,f_j)$ of any two subspaces of Euclidean space such that $(e_i, f_j)=0$ if $i\neq j$ and $(e_i, f_j) \geq 0$

This is a question from a question bank my teacher linked to us.

The way I want to approach it is to let $(e_1,...,e_k)$ be the standard basis and then define some sort of map that will ensure $(e_i, f_j)=0$ $\forall$ $i\neq j$

I'm struggling with being able to come up with this map on my own. Any help would be appreciated.

Nicky_Ay
  • 177
  • 1
    This is not really an easy question. Using the standard basis is not an option, since one is not in $\Bbb R^n$ in the first place and also because these are subspaces. However you can always choose some orthonormal base in one subspace as starting point. – Marc van Leeuwen Mar 28 '15 at 04:49
  • 1
    Euclidean space is $\mathbb{R}^n$, isn't it? – Nicky_Ay Mar 29 '15 at 12:43
  • 1
    No, Euclidean (vector) space is a real vector space (usually assumed finite dimensional) equipped with a positive definite inner product. – Marc van Leeuwen Mar 29 '15 at 17:05

2 Answers2

6

A somewhat slick solution: Let the subspaces be $U$ and $V$, inside Euclidean space $E$. Let $\hat{P}:E \rightarrow V$ denote projection, and $P:U \rightarrow V$ denote the restriction of $\hat{P}$ to $U$. Then $A = P^{T}P:U \rightarrow U$ is symmetric, and so one can form an orthonormal basis $e_1,\ldots,e_k$ of $U$ consisting of eigenvectors of $A$. Order the basis so that the vectors in the null space of $A$ come last. (note that $Nul(A) = Nul(P)$)

Then define $f_i = Pe_i$, for each of the $e_i$ such that $Pe_i$ are nonzero. We have $\langle f_i, f_j \rangle = \langle Pe_i,Pe_j \rangle = \langle e_i, P^{T}Pe_j \rangle = \langle e_i, Ae_j \rangle = \lambda_j \langle e_i, e_j \rangle$, hence the $f_i$ are pairwise orthogonal, and we can complete this to an orthogonal basis of $V$.

It remains to show that $\langle e_i, f_i \rangle \geq 0$, and $\langle e_i, f_j \rangle = 0$ if $i \neq j$.

If $Pe_i = 0$, then $e_i$ is orthogonal to every vector in $V$ and so $\langle e_i, f_j \rangle = 0$ for all $j$.

Otherwise, we have $e_i = f_i + \tilde{f}_i$, where $\tilde{f}_i$ is orthogonal to $V$. Then $\langle e_i, f_j \rangle = \langle f_i + \tilde{f}_i, f_j \rangle = \langle f_i, f_j \rangle$, and the result follows since the $f$'s form an orthogonal basis.

Finally, normalize the $f$'s.

Alex Zorn
  • 3,502
  • 15
  • 17
  • What is $P^T$? Adjoints are only defined for maps from an inner product space to itself, no? So your first and second paragraphs don't really make sense to me. – shalop Mar 28 '15 at 05:46
  • @Shalop: Adjoints are defined for any linear map between inner product spaces. In the current situation $P$ is an orthogonal projection of $U\to V$, and (especially to avoid confusion coming from the fact that orthogonal projections as endomorphisms are self-adjoint) it might have been clearer to identify $P^T$ as the orthogonal projection $\pi:V\to U$. Proof: $(u,\pi(v))=(u,v)=(P(u),v)$ for any $u\in U,v\in V$. – Marc van Leeuwen Mar 28 '15 at 10:59
  • Minor point: the set of vectors $f_i$ is orthonormal, but not a complete basis of$~V$ in general. Extending it to an orthonormal basis is easy though, and the extra vectors will be $\perp U$. – Marc van Leeuwen Mar 28 '15 at 11:01
  • Okay, I understand – shalop Mar 28 '15 at 18:14
  • @MarcvanLeeuwen Whenever I see something like "And then define __ such that __" as part of an answer, I get really confused. What tells you that you should define such $f_i$? I understand the rest of your steps. – Nicky_Ay Mar 29 '15 at 01:31
  • Sorry I meant to tag @Alex Zorn – Nicky_Ay Mar 29 '15 at 01:58
  • @Nicky_Ay: (Excuse my answering for Alex Zorn again): that particular "such that" just means "for which": this definition of $f_i$ is done only for those$~i$ for which $Pe_i\neq0$. That condition is equivalent to $Ae_i\neq0$ (easy proof). Since the $e_i$ were ordered such that those in $\ker(A)$ come last, one therefore defines $f_i$ for an initial subset of indices $i$. Compare with my answer, which says essentially the same things in slightly different wording: "If there are $k$ such vectors, the first $k$ vectors $f_i$...". – Marc van Leeuwen Mar 29 '15 at 03:39
2

Call the two subspaces $V,W$, and call $p_V,p_W$ the orthogonal projections onto these subspaces.

The question is not easy, since not all vectors are candidate for being element of such a basis. If $v\in V$ is such that $p_W(v)\neq 0$, then taking $p=e_i$ forces $f_i$ to be a scalar multiple of $p_W(v)$, as it must be perpendicular to all other $f_j\in W$, which are perpendicular to $v$ and therefore to $p_W(v)$. But similarly $v=e_i$ must be a scalar multiple of$~p_V(f_i)$, and one concludes that $v$ must be an eigenvector for the linear operator $(p_V\circ p_W)|_V$ on $V$.

Given this, it is not surprising that one should invoke the spectral theorem. Let $B$ be the bilinear form on$~V$ defined by $B(v,v')=(p_W(v),p_W(v'))$. (Alternative expressions for $B(v,v')$ are $(p_W(v),v')$ or $(p_V(p_W(v)),v')$.) This bilinear form is clearly symmetric, so by the spectral theorem there exists an orthonormal basis of $V$ which is also orthogonal for$~B$ (this is the same thing as an orthonormal basis of eigenvectors for $(p_V\circ p_W)|_V$). Since $0\leq B(v,v)\leq(v,v)$ for all$~V$, the eigenvalues of $(p_V\circ p_W)|_V$ lie in the interval $[0,1]$, in particular they are non-negative. Now one can take the $e_i$ to run through such a basis, starting with those that have nonzero eigenvalues for $(p_V\circ p_W)|_V$, or what amounts to the same, nonzero images by$~p_W$. If there are $k$ such vectors, the first $k$ vectors $f_i$ can be taken to be $p_W(e_i)$ normalised to unit length by a positive scalar. One can complete these to an orthonormal basis of $W$ with any orthonormal basis of $W\cap V^\perp$. Checking the required properties is easy (and done in the answer by Alex Zorn).

Marc van Leeuwen
  • 115,048