Linear Algebra and Its Applications, Exercise 3.2.16

Exercise 3.2.16. a) Given the projection matrix P projecting vectors onto the line through a and two vectors x and y, show that the inner products of x with Py and y with Px are equal.

b) In general would the angles between x and a and y and a be equal to each other? If a = \left(1, 1, -1\right), x = \left(2, 0, 1\right), and y = \left(2, 1, 2\right) what are the cosines of the two angles?

c) Show that the inner product of Px and Py is the same as the inner products of x with Py and y with Px and explain why this is. What is the angle between the vectors Px and Py?

Answer: a) We have P = aa^T/a^Ta so that

Px = \left(aa^T/a^Ta\right)x = \frac{1}{a^Ta}\left(aa^Tx\right) = \frac{a^Tx}{a^Ta}a

and

Py = \left(aa^T/a^Ta\right)y = \frac{1}{a^Ta}\left(aa^Ty\right) = \frac{a^Ty}{a^Ta}a

The inner product of x with Py is then

x^T\left(Py\right) = x^T\left(a^Ty/a^Ta\right)a

= \left(a^Ty/a^Ta\right)x^Ta = \left(a^Tyx^Ta\right)/a^Ta

= \left(a^Tya^Tx\right)/a^Ta = \left(a^Txa^Ty\right)/a^Ta

and the inner product of y with Px is then

y^T\left(Px\right) = y^T\left(a^Tx/a^Ta\right)a

= \left(a^Tx/a^Ta\right)y^Ta = \left(a^Txy^Ta\right)/a^Ta

= \left(a^Txa^Ty\right)/a^Ta

So the inner products of x with Py and y with Px are equal.

b) Since x and y are arbitrary vectors, in general they would not make the same angle with respect to a. For example, consider the case when a = \left(1, 1, -1\right), x = \left(2, 0, 1\right), and y = \left(2, 1, 2\right).

If \theta_1 is the angle between x and a then \cos \theta_1 = a^Tx/\left(\|a\|\|x\|\right). Similarly, if \theta_2 is the angle between y and a then \cos \theta_2 = a^Ty/\left(\|a\|\|y\|\right)

In this case we have

a^Tx = 1 \cdot 2 + 1 \cdot 1 - 1 \cdot 2 = 2 + 1 - 2 = 1

a^Ty = 1 \cdot 2 + 1 \cdot 0 - 1 \cdot 1 = 2 + 0 - 1 = 1

\|a\| = \sqrt{1^2 + 1^2 + \left(-1\right)^2} = \sqrt{1 + 1 + 1} = \sqrt{3}

\|x\| = \sqrt{2^2 + 0^2 + 1^2} = \sqrt{4 + 0 + 1} = \sqrt{5}

\|y\| = \sqrt{2^2 + 1^2 + 2^2} = \sqrt{4 + 1 + 4} = \sqrt{9} = 3

So \cos \theta_1 = \frac{1}{\sqrt{3}\sqrt{5}} = \frac{1}{\sqrt{15}} and \cos \theta_2 = \frac{1}{\sqrt{3} \cdot 3} = \frac{1}{3\sqrt{3}}. The two cosines are different and thus \theta_1 and \theta_2 are not equal to each other.

c) As noted above we have Px = \frac{a^Tx}{a^Ta}a and Py = \frac{a^Ty}{a^Ta}a. Their inner product is then

\left(Px\right)^T\left(Py\right) = \left(\frac{a^Tx}{a^Ta}a\right)^T \left(\frac{a^Ty}{a^Ta}a\right)

\frac{a^Tx}{a^Ta} \frac{a^Ty}{a^Ta} \left(a^Ta\right) = \left(a^Txa^Ty\right)/a^Ta

But this is the same as the inner products x^TPy and y^TPx of x with Py and y with Px respectively.

This can be understood geometrically as follows: The vector x can be thought of as consisting of two components p_x and q_x where p_x is the projection of x onto a and q_x is the projection of x onto the vector b that is orthogonal to a. The inner product between x and Py is then

\left(Py\right)^Tx = \left(Py\right)^T\left(p_x+q_x\right)

= \left(Py\right)^Tp_x + \left(Py\right)^Tq_x

But p_x is simply Px. Also, since Px is a projection on a and q_x is a projection onto b their inner product is zero since a is orthogonal to b. We therefore have \left(Py\right)^Tx = \left(Py\right)^T\left(Px\right).

Similarly the vector y can be thought of as consisting of two components p_y = Py parallel to a and q_y orthogonal to a, with the inner product between y and Px then being

\left(Px\right)^Ty = \left(Px\right)^T\left(p_y+q_y\right)

= \left(Px\right)^Tp_y + \left(Px\right)^Tq_y

= \left(Px\right)^TPy = \left(Py\right)^TPx

We thus have \left(Py\right)^Tx = \left(Py\right)^TPx = \left(Px\right)^Ty.

Since Px and Py are both projections onto a the angle between them is zero.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books.

 Buy me a snack to sponsor more posts like this!

This entry was posted in linear algebra and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s