Linear Algebra and Its Applications, Exercise 3.2.11

Exercise 3.2.11. a) Given the line through the origin and $a = \left( 1, 3 \right)$ find the matrix $P_1$ that projects onto this line, as well as the matrix $P_2$ that projects onto the line perpendicular to the original line.

b) What is $P_1 + P_2$ ? What is $P_1P_2$ ? Explain your answers.

Answer: a) We have

$P_1 = aa^T/a^Ta = \frac{1}{1^2 + 3^2} \begin{bmatrix} 1 \\ 3 \end{bmatrix}\begin{bmatrix} 1&3 \end{bmatrix} = \frac{1}{10} \begin{bmatrix} 1&3 \\ 3&9 \end{bmatrix}$

We next find the perpendicular line. If this line passes through the origin and $b = \left(b_1, b_2 \right)$ then we will have $b$ orthogonal to $a$ so that $a^Tb = a_1b_1 + a_2b_2 = b_1 + 3b_2 = 0$ . One vector satisfying this condition is $b = \left( -3, 1 \right)$ . We then have

$P_2 = bb^T/b^Tb = \frac{1}{(-3)^2 + 1^2} \begin{bmatrix} -3 \\ 1 \end{bmatrix}\begin{bmatrix} -3&1 \end{bmatrix} = \frac{1}{10} \begin{bmatrix} 9&-3 \\ -3&1 \end{bmatrix}$

b) Taking the product of the two projection matrices first, we have

$P_1P_2 = \frac{1}{10} \begin{bmatrix} 1&3 \\ 3&9 \end{bmatrix} \frac{1}{10} \begin{bmatrix} 9&-3 \\ -3&1 \end{bmatrix}$

$= \frac{1}{100} \begin{bmatrix} 9-9&-3+3 \\ 27-27&-9+9 \end{bmatrix} = \frac{1}{100} \begin{bmatrix} 0&0 \\ 0&0 \end{bmatrix} = 0$

This true because multiplying a vector by $P_2$ projects it onto the line through $b$ , and multiplying by $P_1$ projects the resulting vector onto the line through $a$ . But because $b$ is perpendicular to $a$ in multiplying by $P_1$ all points on the line through $b$ get projected onto the origin. The effect of multiplying by the product matrix $P_1P_2$ is thus the same as multiplying by zero. (Note also that $P_2P_1 = 0$ for the same reason.)

Taking the sum of the two projection matrices, we have

$P_1 + P_2 = \frac{1}{10} \begin{bmatrix} 1&3 \\ 3&9 \end{bmatrix} + \frac{1}{10} \begin{bmatrix} 9&-3 \\ -3&1 \end{bmatrix}$

$= \frac{1}{10} \begin{bmatrix} 1+9&3-3 \\ 3-3&9+1 \end{bmatrix} = \frac{1}{10} \begin{bmatrix} 10&0 \\ 0&10 \end{bmatrix}$

$= \begin{bmatrix} 1&0 \\ 0&1 \end{bmatrix} = I$

In other words, multiplying a vector by the sum of the two projection matrices results in the same vector:

$(P_1+P_2)v = P_1v + P_2v = v$

In effect we have transformed the vector $v$ into two perpendicular component vectors: one component $P_1v$ on the line through $a$ and a second component $P_2v$ on the perpendicular line through $b$ . Adding the components back together reconstitutes the original vector.

UPDATE: Corrected the calculations above to include the factors $1/a^Ta$ and $1/b^Tb$ when calculating the projection matrices. Thanks go to Trystyn for catching my original error.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books.

Buy me a snack to sponsor more posts like this!

2 Responses to Linear Algebra and Its Applications, Exercise 3.2.11

Trystyn says:

October 3, 2014 at 7:23 pm

This is wrong.
P = [a a^T] / [a^T a]
not just [a a^T]

hecker says:

October 4, 2014 at 1:39 pm

Trystyn: You are correct, thanks for finding this error! I’ve updated the post to reflect your correction.