Linear Algebra and Its Applications, Exercise 3.2.16

Exercise 3.2.16. a) Given the projection matrix $P$ projecting vectors onto the line through $a$ and two vectors $x$ and $y$ , show that the inner products of $x$ with $Py$ and $y$ with $Px$ are equal.

b) In general would the angles between $x$ and $a$ and $y$ and $a$ be equal to each other? If $a = \left(1, 1, -1\right)$ , $x = \left(2, 0, 1\right)$ , and $y = \left(2, 1, 2\right)$ what are the cosines of the two angles?

c) Show that the inner product of $Px$ and $Py$ is the same as the inner products of $x$ with $Py$ and $y$ with $Px$ and explain why this is. What is the angle between the vectors $Px$ and $Py$ ?

Answer: a) We have $P = aa^T/a^Ta$ so that

$Px = \left(aa^T/a^Ta\right)x = \frac{1}{a^Ta}\left(aa^Tx\right) = \frac{a^Tx}{a^Ta}a$

and

$Py = \left(aa^T/a^Ta\right)y = \frac{1}{a^Ta}\left(aa^Ty\right) = \frac{a^Ty}{a^Ta}a$

The inner product of $x$ with $Py$ is then

$x^T\left(Py\right) = x^T\left(a^Ty/a^Ta\right)a$

$= \left(a^Ty/a^Ta\right)x^Ta = \left(a^Tyx^Ta\right)/a^Ta$

$= \left(a^Tya^Tx\right)/a^Ta = \left(a^Txa^Ty\right)/a^Ta$

and the inner product of $y$ with $Px$ is then

$y^T\left(Px\right) = y^T\left(a^Tx/a^Ta\right)a$

$= \left(a^Tx/a^Ta\right)y^Ta = \left(a^Txy^Ta\right)/a^Ta$

$= \left(a^Txa^Ty\right)/a^Ta$

So the inner products of $x$ with $Py$ and $y$ with $Px$ are equal.

b) Since $x$ and $y$ are arbitrary vectors, in general they would not make the same angle with respect to $a$ . For example, consider the case when $a = \left(1, 1, -1\right)$ , $x = \left(2, 0, 1\right)$ , and $y = \left(2, 1, 2\right)$ .

If $\theta_1$ is the angle between $x$ and $a$ then $\cos \theta_1 = a^Tx/\left(\|a\|\|x\|\right)$ . Similarly, if $\theta_2$ is the angle between $y$ and $a$ then $\cos \theta_2 = a^Ty/\left(\|a\|\|y\|\right)$

In this case we have

$a^Tx = 1 \cdot 2 + 1 \cdot 1 - 1 \cdot 2 = 2 + 1 - 2 = 1$

$a^Ty = 1 \cdot 2 + 1 \cdot 0 - 1 \cdot 1 = 2 + 0 - 1 = 1$

$\|a\| = \sqrt{1^2 + 1^2 + \left(-1\right)^2} = \sqrt{1 + 1 + 1} = \sqrt{3}$

$\|x\| = \sqrt{2^2 + 0^2 + 1^2} = \sqrt{4 + 0 + 1} = \sqrt{5}$

$\|y\| = \sqrt{2^2 + 1^2 + 2^2} = \sqrt{4 + 1 + 4} = \sqrt{9} = 3$

So $\cos \theta_1 = \frac{1}{\sqrt{3}\sqrt{5}} = \frac{1}{\sqrt{15}}$ and $\cos \theta_2 = \frac{1}{\sqrt{3} \cdot 3} = \frac{1}{3\sqrt{3}}$ . The two cosines are different and thus $\theta_1$ and $\theta_2$ are not equal to each other.

c) As noted above we have $Px = \frac{a^Tx}{a^Ta}a$ and $Py = \frac{a^Ty}{a^Ta}a$ . Their inner product is then

$\left(Px\right)^T\left(Py\right) = \left(\frac{a^Tx}{a^Ta}a\right)^T \left(\frac{a^Ty}{a^Ta}a\right)$

$\frac{a^Tx}{a^Ta} \frac{a^Ty}{a^Ta} \left(a^Ta\right) = \left(a^Txa^Ty\right)/a^Ta$

But this is the same as the inner products $x^TPy$ and $y^TPx$ of $x$ with $Py$ and $y$ with $Px$ respectively.

This can be understood geometrically as follows: The vector $x$ can be thought of as consisting of two components $p_x$ and $q_x$ where $p_x$ is the projection of $x$ onto $a$ and $q_x$ is the projection of $x$ onto the vector $b$ that is orthogonal to $a$ . The inner product between $x$ and $Py$ is then

$\left(Py\right)^Tx = \left(Py\right)^T\left(p_x+q_x\right)$

$= \left(Py\right)^Tp_x + \left(Py\right)^Tq_x$

But $p_x$ is simply $Px$ . Also, since $Px$ is a projection on $a$ and $q_x$ is a projection onto $b$ their inner product is zero since $a$ is orthogonal to $b$ . We therefore have $\left(Py\right)^Tx = \left(Py\right)^T\left(Px\right)$ .

Similarly the vector $y$ can be thought of as consisting of two components $p_y = Py$ parallel to $a$ and $q_y$ orthogonal to $a$ , with the inner product between $y$ and $Px$ then being

$\left(Px\right)^Ty = \left(Px\right)^T\left(p_y+q_y\right)$

$= \left(Px\right)^Tp_y + \left(Px\right)^Tq_y$

$= \left(Px\right)^TPy = \left(Py\right)^TPx$

We thus have $\left(Py\right)^Tx = \left(Py\right)^TPx = \left(Px\right)^Ty$ .

Since $Px$ and $Py$ are both projections onto $a$ the angle between them is zero.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books.

Buy me a snack to sponsor more posts like this!

Linear Algebra and Its Applications, Exercise 3.2.16

Leave a Reply Cancel reply

Archives

Meta

Linear Algebra and Its Applications, Exercise 3.2.16

Share this:

Like this:

Related

Leave a Reply Cancel reply

Archives

Meta