Exercise 3.3.11. Suppose that is a subspace with orthogonal complement , with a projection matrix onto and a projection matrix onto . What are and ? Also, show that is its own inverse.

Answer: Given any vector we have where is the projection of onto and is the projection of onto . Since and are orthogonal complements the sum of the projections and is equal to itself. So we have and since this is true for all we have .

This can be more formally proved as follows: Consider the matrix . It is a projection matrix and it projects onto . Also, if is a projection matrix onto then it is unique. Since is a projection matrix onto and is a projection matrix onto we therefore have or . For the full proof see below.

Since we have

.

(Applying to a vector first projects onto and then projects the resulting vector onto . But projecting any vector in onto will produce the zero vector, since and are orthogonal.)

Finally, we have

Since we have so that is its own inverse.

Here is the full proof that if a projection matrix onto and a projection matrix onto then :

We first show that is a projection matrix. The identity matrix is symmetric, and since is a projection matrix it is also symmetric. The difference is therefore symmetric as well, so that we have .

We also have

Since and we see that is a projection matrix.

Onto what subspace does project? For any vector we have . Since is a projection matrix the vector is in the space onto which projects, and the vector is orthogonal to that space. But projects onto so must therefore be in .

We have thus shown that is a projection matrix that projects onto . We now show that any such projection matrix is unique.

Suppose that like the matrix is also a projection matrix onto . Consider the vector for any vector . We have

where we take advantage of the fact that .

But since and are projection matrices they are symmetric, and therefore their difference is also symmetric. We thus have

But since and are projection matrices we have and . We thus have

Since is a projection matrix onto we know that for any vector in . (In other words, when applied to any vector in the projection matrix projects that vector onto itself.) The same is true for since it is also a projection matrix onto . We thus have for all in .

Given any vector (not necessarily in ) we then have since is a vector in . Since is a projection matrix we have and thus , so that for all vectors . This implies that .

Similarly for any vector we have since is a vector in . Since is a projection matrix we have and thus , so that for all vectors . This implies that .

Since and we have

Since we have , and since this is true for any vector we must have or . A projection matrix onto a subspace is therefore unique.

Thus since is a projection matrix onto and is also a projection matrix onto we have or .

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.