Linear Algebra and Its Applications, Exercise 3.3.11

Exercise 3.3.11. Suppose that $S$ is a subspace with orthogonal complement $S^{\perp}$ , with $P$ a projection matrix onto $S$ and $Q$ a projection matrix onto $S^{\perp}$ . What are $P+Q$ and $PQ$ ? Also, show that $P-Q$ is its own inverse.

Answer: Given any vector $v$ we have $(P+Q)v = Pv + Qv$ where $Pv$ is the projection of $v$ onto $S$ and $Qv$ is the projection of $v$ onto $S^{\perp}$ . Since $S$ and $S^\perp$ are orthogonal complements the sum of the projections $Pv$ and $Qv$ is equal to $v$ itself. So we have $(P+Q)v = Pv + Qv = v$ and since this is true for all $v$ we have $P+Q = I$ .

This can be more formally proved as follows: Consider the matrix $I-Q$ . It is a projection matrix and it projects onto $S$ . Also, if $P$ is a projection matrix onto $S$ then it is unique. Since $P$ is a projection matrix onto $S$ and $I-Q$ is a projection matrix onto $S$ we therefore have $P = I-Q$ or $P+Q = I$ . For the full proof see below.

Since $P = I - Q$ we have

$PQ = (I - Q)Q = IQ - Q^2 = Q - Q = 0$ .

(Applying $PQ$ to a vector $v$ first projects $v$ onto $S^\perp$ and then projects the resulting vector onto $S$ . But projecting any vector in $S^\perp$ onto $S$ will produce the zero vector, since $S^\perp$ and $S$ are orthogonal.)

Finally, we have

$(P-Q)(P-Q) = [(I-Q) - Q][(I-Q) - Q]$

$(I - 2Q)(I - 2Q) = I^2 - 2Q - 2Q + 4Q^2$

$I - 4Q + 4Q = I$

Since $(P-Q)(P-Q) = I$ we have $(P-Q)^{-1} = P-Q$ so that $P-Q$ is its own inverse.

Here is the full proof that if $P$ a projection matrix onto $S$ and $Q$ a projection matrix onto $S^{\perp}$ then $P+Q = I$ :

We first show that $I-Q$ is a projection matrix. The identity matrix $I$ is symmetric, and since $Q$ is a projection matrix it is also symmetric. The difference $I-Q$ is therefore symmetric as well, so that we have $(I-Q)^T = I-Q$ .

We also have

$(I-Q)^2 = (I-Q)(I-Q) = I^2 - IQ - QI + Q^2$

$= I - 2Q + Q = I-Q$

Since $(I-Q)^T = I-Q$ and $(I-Q)^2 = I-Q$ we see that $I-Q$ is a projection matrix.

Onto what subspace does $I-Q$ project? For any vector $v$ we have $(I-Q)v = v - Qv$ . Since $Q$ is a projection matrix the vector $Qv$ is in the space onto which $Q$ projects, and the vector $v - Qv$ is orthogonal to that space. But $Q$ projects onto $S^\perp$ so $v-Qv$ must therefore be in $(S^\perp)^\perp = S$ .

We have thus shown that $I-Q$ is a projection matrix that projects onto $S$ . We now show that any such projection matrix is unique.

Suppose that like $P$ the matrix $P'$ is also a projection matrix onto $S$ . Consider the vector $(P - P')v$ for any vector $v$ . We have

$\|(P - P')v\|^2 = [(P-P')v]^T[(P-P')v]$

$= v^T(P-P')^T(P-P')v$

where we take advantage of the fact that $(AB)^T = B^TA^T$ .

But since $P$ and $P'$ are projection matrices they are symmetric, and therefore their difference $P-P'$ is also symmetric. We thus have

$\|(P - P')v\|^2 = v^T(P-P')(P-P')v$

$= v^T[P^2 - PP' - P'P + (P')^2]v$

But since $P$ and $P'$ are projection matrices we have $P = P^2$ and $P' = (P')^2$ . We thus have

$\|(P - P')v\|^2 = v^T[P - PP' - P'P + P']v$

Since $P$ is a projection matrix onto $S$ we know that $Pw = w$ for any vector $w$ in $S$ . (In other words, when applied to any vector $w$ in $S$ the projection matrix $P$ projects that vector onto itself.) The same is true for $P'$ since it is also a projection matrix onto $S$ . We thus have $Pw = P'w = w$ for all $w$ in $S$ .

Given any vector $v$ (not necessarily in $S$ ) we then have $PPv = P'Pv$ since $Pv$ is a vector in $S$ . Since $P$ is a projection matrix we have $P^2 = P$ and thus $PPv = Pv$ , so that $Pv = P'Pv$ for all vectors $v$ . This implies that $P = P'P$ .

Similarly for any vector $v$ we have $P'P'v = PP'v$ since $P'v$ is a vector in $S$ . Since $P'$ is a projection matrix we have $(P')^2 = P'$ and thus $P'P'v = P'v$ , so that $P'v = PP'v$ for all vectors $v$ . This implies that $P' = PP'$ .

Since $P = P'P$ and $P' = PP'$ we have

$\|(P - P')v\|^2 = v^T(P - P' - P + P')v$

$= v^T \cdot 0 \cdot v = 0$

Since $\|(P - P')v\|^2 = 0$ we have $(P-P')v = 0$ , and since this is true for any vector $v$ we must have $P-P' = 0$ or $P = P'$ . A projection matrix $P$ onto a subspace $S$ is therefore unique.

Thus since $P$ is a projection matrix onto $S$ and $I-Q$ is also a projection matrix onto $S$ we have $P = I-Q$ or $P+Q=I$ .

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Linear Algebra and Its Applications, Exercise 3.3.11

Leave a Reply Cancel reply

Archives

Meta

Linear Algebra and Its Applications, Exercise 3.3.11

Share this:

Like this:

Related

Leave a Reply Cancel reply

Archives

Meta