## Linear Algebra and Its Applications, Exercise 3.4.10

Exercise 3.4.10. Given the two orthonormal vectors $q_1$ and $q_2$ and an arbitrary vector $b$, what linear combination of $q_1$ and $q_2$ is the least distance from $b$? Show that the difference between $b$ and that combination (i.e., the error vector) is orthogonal to both $q_1$ and $q_2$.

Answer: This exercise is similar to the previous one. Any linear combination of $q_1$ and $q_2$ is in the plane formed by $q_1$ and $q_2$, and the combination closest to $b$ is the projection $p$ of $b$ onto that plane. Because $q_1$ and $q_2$ are orthonormal that projection is equal to the sum of the separate projections of $b$ onto $q_1$ and $q_2$ respectively: $p = (q_1^Tb)q_1 + (q_2^Tb)q_2$

So $(q_1^Tb)q_1 + (q_2^Tb)q_2$ is the closest combination to $b$.

The error vector is then $e = b - p = b - (q_1^Tbq_1 + q_2^Tbq_2)$

Taking the dot product of the error vector with $q_1$ we have $q_1^Te = q_1^T [b - (q_1^Tbq_1 + q_2^Tbq_2)] = q_1^Tb - q_1^T(q_1^Tb)q_1 - q_1^T(q_2^Tb)q_2$ $= q_1^Tb - (q_1^Tb)q_1^Tq_1 - (q_2^Tb)q_1^Tq_2$

Since $q_1$ and $q_2$ are orthonormal we have $q_1^Tq_1 = 1$ and $q_1^Tq_2 = 0$. So we have $q_1^Te = q_1^Tb - (q_1^Tb) \cdot 1 - (q_2^Tb) \cdot 0 = q_1^Tb - q_1^Tb = 0$

Similarly we have $q_2^Te = q_2^T [b - (q_1^Tbq_1 + q_2^Tbq_2)] = q_2^Tb - q_2^T(q_1^Tb)q_1 - q_2^T(q_2^Tb)q_2$ $= q_2^Tb - (q_1^Tb)q_2^Tq_1 - (q_2^Tb)q_2^Tq_2 = q_2^Tb - q_2^Tb = 0$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , . Bookmark the permalink.