## Linear Algebra and Its Applications, Exercise 3.3.5

Exercise 3.3.5. Given the system $Ax = \begin{bmatrix} 1&-1 \\ 1&0 \\ 1&1 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} 4 \\ 5 \\ 9 \end{bmatrix} = b$

with no solution, provide a graph of a straight line that minimizes $(C - D -4)^2 + (C - 5)^2 + (C + D - 9)^2$

and solve for the equation of the line. What is the result of projecting the vector $b$ onto the column space of $A$? $(C - D -4)^2 + (C - 5)^2 + (C + D - 9)^2 = \| Ax - b \|^2$

In other words, this problem is essentially that of finding $\bar{x} = \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix}$, the least squares solution to $Ax = b$ that minimizes the error vector $E^2 = \| Ax - b \|^2$.

From 3L on page 156 we have $\bar{x} = (A^TA)^{-1}A^Tb$. In this case we have $A^TA = \begin{bmatrix} 1&1&1 \\ -1&0&1 \end{bmatrix} \begin{bmatrix} 1&-1 \\ 1&0 \\ 1&1 \end{bmatrix} = \begin{bmatrix} 3&0 \\ 0&2 \end{bmatrix}$

and thus $(A^TA)^{-1} = \frac{1}{6} \begin{bmatrix} 2&0 \\ 0&3 \end{bmatrix}$

so that $\bar{x} = (A^TA)^{-1}A^Tb = \frac{1}{6} \begin{bmatrix} 2&0 \\ 0&3 \end{bmatrix} \begin{bmatrix} 1&-1 \\ 1&0 \\ 1&1 \end{bmatrix} \begin{bmatrix} 4 \\ 5 \\ 9 \end{bmatrix}$ $= \frac{1}{6} \begin{bmatrix} 2&0 \\ 0&3 \end{bmatrix} \begin{bmatrix} 18 \\ 5 \end{bmatrix} = \frac{1}{6} \begin{bmatrix} 36 \\ 15 \end{bmatrix} = \begin{bmatrix} 6 \\ \frac{5}{2} \end{bmatrix}$

Thus the line that minimizes $E^2 = \|Ax -b\|^2$ $= (C - D - 4)^2 + (C - 5)^2 + (C + D - 9)^2$

is $\bar{C} + \bar{D}t$ with  slope $\bar{D} = \frac{5}{2}$ and intercept $\bar{C} = 6$.

Also from 3L on page 156 the projection of $b$ onto the column space of $A$ is $p = A\bar{x} = \begin{bmatrix} 1&-1 \\ 1&0 \\ 1&1 \end{bmatrix} \begin{bmatrix} 6 \\ \frac{5}{2} \end{bmatrix} = \begin{bmatrix} \frac{7}{2} \\ 6 \\ \frac{17}{2} \end{bmatrix}$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , , , . Bookmark the permalink.