## Linear Algebra and Its Applications, Exercise 3.3.6

Exercise 3.3.6. Given the matrix $A$ and vector $b$ defined as follows $A = \begin{bmatrix} 1&1 \\ 2&-1 \\ -2&4 \end{bmatrix} \qquad b = \begin{bmatrix} 1 \\ 2 \\ 7 \end{bmatrix}$

find the projection of $b$ onto the column space of $A$.

Decompose the vector $b$ into the sum $p + q$ of two orthogonal vectors $p$ and $q$ where $p$ is in the column space. Which subspace is $q$ in?

Answer: We have $p = A(A^TA)^{-1}A^Tb$ per equation (3) of 3L on page 156. We first compute $A^TA = \begin{bmatrix} 1&2&-2 \\ 1&-1&4 \end{bmatrix} \begin{bmatrix} 1&1 \\ 2&-1 \\ -2&4 \end{bmatrix}$ $= \begin{bmatrix} 9&-9 \\ -9&18 \end{bmatrix}$

and then compute $(A^TA)^{-1} = \frac{1}{9 \cdot 18 - (-9)(-9)} \begin{bmatrix} 18&-(-9) \\ -(-9)&9 \end{bmatrix}$ $= \frac{1}{81} \begin{bmatrix} 18&9 \\ 9&9 \end{bmatrix} = \begin{bmatrix} \frac{2}{9}&\frac{1}{9} \\ \frac{1}{9}&\frac{1}{9} \end{bmatrix}$

Finally we compute $p = A(A^TA)^{-1}A^Tb$ $= \begin{bmatrix} 1&1 \\ 2&-1 \\ -2&4 \end{bmatrix} \begin{bmatrix} \frac{2}{9}&\frac{1}{9} \\ \frac{1}{9}&\frac{1}{9} \end{bmatrix} \begin{bmatrix} 1&2&-2 \\ 1&-1&4 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \\ 7 \end{bmatrix}$ $= \begin{bmatrix} 1&1 \\ 2&-1 \\ -2&4 \end{bmatrix} \begin{bmatrix} \frac{2}{9}&\frac{1}{9} \\ \frac{1}{9}&\frac{1}{9} \end{bmatrix} \begin{bmatrix} -9 \\ 27 \end{bmatrix}$ $= \begin{bmatrix} 1&1 \\ 2&-1 \\ -2&4 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} 3 \\ 0 \\ 6 \end{bmatrix}$

Since $b = p + q$ we have $q = b - p = \begin{bmatrix} 1 \\ 2 \\ 7 \end{bmatrix} - \begin{bmatrix} 3 \\ 0 \\ 6 \end{bmatrix} = \begin{bmatrix} -2 \\ 2 \\ 1 \end{bmatrix}$

We have $p \cdot q = 3 \cdot (-2) + 0 \cdot 2 + 6 \cdot 1 = -6 + 0 + 6 = 0$

so that $p$ and $q$ are orthogonal.

The vector $p$ is in $\cal{R}(A)$, the column space of A, and the orthogonal subspace of $\cal{R}(A)$ is $\cal{N}(A^T)$, the left nullspace of $A$. Since $p$ is in $\cal{R}(A)$ and $q$ is orthogonal to $p$, $q$ must be in $\cal{N}(A^T)$, so that $A^Tq = 0$. We confirm this: $A^Tq = \begin{bmatrix} 1&2&-2 \\ 1&-1&4 \end{bmatrix} \begin{bmatrix} -2 \\ 2 \\ 1 \end{bmatrix}$ $= \begin{bmatrix} -2 + 4 -2 \\ -2 - 2 + 4 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \end{bmatrix} = 0$

UPDATE: I corrected the calculation of $q$; thanks go to KTL for pointing out the error.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , , , , . Bookmark the permalink.

### 6 Responses to Linear Algebra and Its Applications, Exercise 3.3.6

1. Teresa says:

Thank you for the great work! It’s been really helpful. I wanted to mention that A transpose by A is [6 -8; -8 18] and not [9 -9; -9 18] that changes the results a bit.

2. hecker says:

I’m glad you find these posts useful. I’m sorry it’s been a long time since I published the last one.

I don’t understand your comment. Are you referring to A transpose multiplied by A (on the right)? If so, the (1, 1) element of the result matrix is 1 x 1 + 2 x 2 + (-2) x (-2) = 1 + 4 + 4 = 9. I don’t know how you got 6 for that result. Similarly the (1, 2) entry is 1 x 1 + 2 x (-1) + (-2) x 4 = 1 – 2 – 8 = -9 (not -8) and the (2, 1) entry is 1 x 1 + (-1) x 2 + 4 x (-2) = 1 – 2 – 8 = -9 again.

• KTL says:

The question in the 4th edition is different. Hence the confusion for the person above 🙂

• hecker says:

Ah, thanks for the explanation. I’ve never looked at a copy of the 4th edition so I don’t know how the exercises differ.

3. KTL says:

q you have given as p-b though the calculation is done as b-p (correctly)

• hecker says:

Thanks for finding this error! I’ve corrected the post.