## Linear Algebra and Its Applications, Exercise 3.4.15

Exercise 3.4.15. Given the matrix $A = \begin{bmatrix} 1&1 \\ 2&-1 \\ -2&4 \end{bmatrix}$

find the orthonormal vectors $q_1$ and $q_2$ that span the column space of $A$. Next find the vector $q_3$ that completes the orthonormal set, and describe the subspace of $A$ of which $q_3$ is an element. Finally, for $b = (1, 2, 7)$ find the least squares solution $\bar{x}$ to $Ax = b$.

Answer: With $a$ and $b$ as the columns of $A$, we first choose $a' = a = (1, 2, -2)$. We then have $b' = b - \frac{(a')^Tb}{(a')^Ta'}a' = b - \frac{1 \cdot 1 + 2 \cdot (-1) + (-2) \cdot 4}{1^2 + 2^2 + (-2)^2}a' = b - \frac{-9}{9}a' = b + a'$ $= \begin{bmatrix} 1 \\ -1 \\ 4 \end{bmatrix} + \begin{bmatrix} 1 \\ 2 \\ -2 \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix}$

Now that we have calculated the orthogonal vectors $a'$ and $b'$ we can normalize them to create the orthonormal vectors $q_1$ and $q_2$. We have $\|a'\| = \sqrt{1^2 + 2^2 + (-2)^2} = \sqrt{9} = 3$ $\|b'\| = \sqrt{2^2 + 1^2 + 2^2} = \sqrt{9} = 3$

so that $q_1 = a' / \|a'\| = \frac{1}{3} \begin{bmatrix} 1 \\ 2 \\ -2 \end{bmatrix} = \begin{bmatrix} \frac{1}{3} \\ \frac{2}{3} \\ -\frac{2}{3} \end{bmatrix}$ $q_2 = b' / \|b'\| = \frac{1}{3} \begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix} = \begin{bmatrix} \frac{2}{3} \\ \frac{1}{3} \\ \frac{2}{3} \end{bmatrix}$

Since $a$ and $b$ span the column space of $A$, and the orthonormal vectors $q_1$ and $q_2$ are linear combinations of $a$ and $b$, $q_1$ and $q_2$ also span the column space of $A$.

Next, we calculate $q_3$. We can do this by orthogonalizing any vector $c$ that is linearly independent of $a$ and $b$. For ease of calculation we start with $c = (1, 0, 0)$. We then have $c' = c - \frac{(a')^Tc}{(a')^Ta'}a' - \frac{(b')^Tc}{(b')^Tb'}b' = c - \frac{1 \cdot 1 + 2 \cdot 0 + (-2) \cdot 0}{1^2 + 2^2 + (-2)^2}a' - \frac{2 \cdot 1 + 1 \cdot 0 + 2 \cdot 0}{1^2 + 2^2 + (-2)^2}b' = c - \frac{1}{9}a' - \frac{2}{9}b'$ $= \begin{bmatrix} 1 \\ 0 \\ 0 \end{bmatrix} - \frac{1}{9} \begin{bmatrix} 1 \\ 2 \\ -2 \end{bmatrix} - \frac{2}{9} \begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix} = \begin{bmatrix} \frac{4}{9} \\ -\frac{4}{9} \\ -\frac{2}{9} \end{bmatrix}$

To normalize $c'$ we divide by $\|c'\| = \sqrt{(-\frac{4}{9})^2 + (-\frac{4}{9})^2 + (-\frac{2}{9})^2} = \sqrt{\frac{36}{81}} = \frac{2}{3}$

so that $q_3 = c' / \|c'\| = \frac{3}{2} \begin{bmatrix} \frac{4}{9} \\ -\frac{4}{9} \\ -\frac{2}{9} \end{bmatrix} = \begin{bmatrix} \frac{2}{3} \\ -\frac{2}{3} \\ -\frac{1}{3} \end{bmatrix}$

Of the four fundamental subspaces of $A$, the left nullspace $\mathcal{N}(A^T)$ is orthogonal to the column space $\mathcal{R}(A)$. Since $q_1$ and $q_2$ span the column space $\mathcal{R}(A)$ and $q_3$ is orthogonal to $q_1$ and $q_2$, $q_3$ must be an element of the left nullspace $\mathcal{N}(A^T)$.

Finally, to find the least squares solution $\bar{x}$ to $Ax = b$ where $b = (1, 2, 7)$, we factor $A = QR$ and take advantage of the fact that $R\bar{x} = Q^Tb$.

The matrix $Q$ is simply the 3 by 2 matrix with columns $q_1$ and $q_2$: $Q = \begin{bmatrix} \frac{1}{3}&\frac{2}{3} \\ \frac{2}{3}&\frac{1}{3} \\ -\frac{2}{3}&\frac{2}{3} \end{bmatrix}$

The upper triangular matrix $R$ is a 2 by 2 matrix with $R = \begin{bmatrix} q_1^Ta&q_1^Tb \\ 0&q_2^Tb \end{bmatrix} = \begin{bmatrix} \frac{1}{3} \cdot 1 + \frac{2}{3} \cdot 2 + (-\frac{2}{3}) \cdot (-2)&\frac{1}{3} \cdot 1 + \frac{2}{3} \cdot (-1) + (-\frac{2}{3}) \cdot 4 \\ 0&\frac{2}{3} \cdot 1 + \frac{1}{3} \cdot (-1) + \frac{2}{3} \cdot 4 \end{bmatrix}$ $= \begin{bmatrix} 3&-3 \\ 0&3 \end{bmatrix}$

On the right side of the equation $R\bar{x} = Q^Tb$ we have $Q^Tb = \begin{bmatrix} \frac{1}{3}&\frac{2}{3}&-\frac{2}{3} \\ \frac{2}{3}&\frac{1}{3}&\frac{2}{3} \end{bmatrix} \begin{bmatrix} 1 \\ 2 \\ 7 \end{bmatrix} = \begin{bmatrix} -\frac{9}{3} \\ \frac{18}{3} \end{bmatrix} = \begin{bmatrix} -3 \\ 6 \end{bmatrix}$

so that the entire system is then $\begin{bmatrix} 3&-3 \\ 0&3 \end{bmatrix} \begin{bmatrix} \bar{x_1} \\ \bar{x_2} \end{bmatrix} = \begin{bmatrix} -3 \\ 6 \end{bmatrix}$

From the second equation we have $3\bar{x_2} = 6$ or $\bar{x_2} = 2$. Substituting into the first equation we have $3\bar{x_1} - 3\bar{x_2} = 3\bar{x_1} - 3 \cdot 2 = 3$ so that $3\bar{x_1} = 3 + 6 = 9$ or $\bar{x_1} = 3$.

The least squares solution to $Ax = b$ with $b = (1, 2, 7)$ is therefore $\bar{x} = (3, 2)$.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , , , , . Bookmark the permalink.