## Linear Algebra and Its Applications, Exercise 3.3.3

Exercise 3.3.3. Given $A = \begin{bmatrix} 1&0 \\ 0&1 \\ 1&1 \end{bmatrix} \qquad b = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}$

solve $Ax = b$ to find $p = A\bar{x}$. Show that $b-p$ is orthogonal to every column in $A$.

Answer: We have $A^TA\bar{x} = A^Tb$ or $\bar{x} = \left(A^TA\right)^{-1}A^Tb$ if $A^TA$ is invertible. In this case we have $A^TA = \begin{bmatrix} 1&0&1 \\ 0&1&1 \end{bmatrix} \begin{bmatrix} 1&0 \\ 0&1 \\ 1&1 \end{bmatrix} = \begin{bmatrix} 2&1 \\ 1&2 \end{bmatrix}$

so that $\left(A^TA\right)^{-1} = \frac{1}{2 \cdot 2 - 1 \cdot 1} \begin{bmatrix} 2&-1 \\ -1&2 \end{bmatrix}$ $= \frac{1}{3} \begin{bmatrix} 2&-1 \\ -1&2 \end{bmatrix} = \begin{bmatrix} \frac{2}{3}&-\frac{1}{3} \\ -\frac{1}{3}&\frac{2}{3} \end{bmatrix}$

We then have $\bar{x} = \begin{bmatrix} \frac{2}{3}&-\frac{1}{3} \\ -\frac{1}{3}&\frac{2}{3} \end{bmatrix} \begin{bmatrix} 1&0&1 \\ 0&1&1 \end{bmatrix} \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix}$ $= \begin{bmatrix} \frac{2}{3}&-\frac{1}{3} \\ -\frac{1}{3}&\frac{2}{3} \end{bmatrix} \begin{bmatrix} 1 \\ 1 \end{bmatrix} = \begin{bmatrix} \frac{1}{3} \\ \frac{1}{3} \end{bmatrix}$

and $p = A\bar{x} = \begin{bmatrix} 1&0 \\ 0&1 \\ 1&1 \end{bmatrix} \begin{bmatrix} \frac{1}{3} \\ \frac{1}{3} \end{bmatrix} = \begin{bmatrix} \frac{1}{3} \\ \frac{1}{3} \\ \frac{2}{3} \end{bmatrix}$

The error vector is then $b - p = \begin{bmatrix} 1 \\ 1 \\ 0 \end{bmatrix} - \begin{bmatrix} \frac{1}{3} \\ \frac{1}{3} \\ \frac{2}{3} \end{bmatrix} = \begin{bmatrix} \frac{2}{3} \\ \frac{2}{3} \\ -\frac{2}{3} \end{bmatrix}$

The inner product of the error vector $b-p$ with column 1 of $A$ is $\begin{bmatrix} \frac{2}{3}&\frac{2}{3}&-\frac{2}{3} \end{bmatrix} \begin{bmatrix} 1 \\ 0 \\ 1 \end{bmatrix} = \frac{2}{3} + 0 - \frac{2}{3} = 0$

The inner product of $b-p$ with column 2 of $A$ is $\begin{bmatrix} \frac{2}{3}&\frac{2}{3}&-\frac{2}{3} \end{bmatrix} \begin{bmatrix} 0 \\ 1 \\ 1 \end{bmatrix} = 0 + \frac{2}{3} - \frac{2}{3} = 0$

Thus $b-p$ is othogonal to all columns of $A$ (and thus to the column space of $A$ as well).

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , , . Bookmark the permalink.