## Linear Algebra and Its Applications, Exercise 3.4.1

Exercise 3.4.1. a) Given the following four data points: $y = -4 \quad\textrm{at}\quad t = -2 \quad\textrm{and}\quad y = -3 \quad\textrm{at}\quad t = -1$ $y = -1 \quad\textrm{at}\quad t = 1 \quad\textrm{and}\quad y = 0 \quad\textrm{at}\quad t = 2$

write down the four equations for fitting $C + Dt$ to the data.

b) Find the line fit by least squares and calculate the error $E^2$.

c) Given the value of $E^2$ what is $b$ in relation to the column space? What is the projection $p$ of $b$ on the column space?

Answer: a) This corresponds to a system of the form $Ax = b$ as follows: $\begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&1 \\ 1&2 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} -4 \\ -3 \\ -1 \\ 0 \end{bmatrix}$

The equivalent system of equations is: $C - 2D = -4$ $C - D = -3$ $C + D = -1$ $C + 2D = 0$

b) To find the least squares solution we multiply both sides by $A^T$ to create a system of the form $A^TA\bar{x} = A^Tb$. We have $A^TA = \begin{bmatrix} 1&1&1&1 \\ -2&-1&1&2 \end{bmatrix} \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&1 \\ 1&2 \end{bmatrix} = \begin{bmatrix} 4&0 \\ 0&10 \end{bmatrix}$

and $A^Tb = \begin{bmatrix} 1&1&1&1 \\ -2&-1&1&2 \end{bmatrix} \begin{bmatrix} -4 \\ -3 \\ -1 \\ 0 \end{bmatrix} = \begin{bmatrix} -8 \\ 10 \end{bmatrix}$

so that the new system is $\begin{bmatrix} 4&0 \\ 0&10 \end{bmatrix} \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = \begin{bmatrix} -8 \\ 10 \end{bmatrix}$

From the second equation we have $\bar{D} = \frac{10}{10} = 1$ and from the first equation we have $\bar{C} = \frac{-8}{4} = -2$.

The resulting graph is a line $-2 + t$ with slope of 1 and $y$-intercept of -2. For the values of $t$ of -2, -1, 1, and 2 the values of $-2 + t$ are -4, -3, -1, and 0 respectively. But these are exactly the same as the values of $y$ in the given data points. Therefore we have $E^2 = 0$.

c) Since $E^2 = 0$ the vector $b$ must be in the column space of $A$, and the projection $p$ of $b$ onto the column space is simply $b$ itself.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , , . Bookmark the permalink.