## Linear Algebra and Its Applications, Exercise 3.3.2

Exercise 3.3.2. We have the value $b_1 = 1$ at time $t_1 = 1$ and the value $b_2 = 7$ at time $t_2 = 2$. We wish to fit these values using a line constrained to go through the origin, i.e., with an equation of the form $b = Dt$. Solve the system using least squares and describe the best fit line.

Answer: In general we would fit a line of the form $C + Dt = b$; however since we want the fitted line to go through the origin we have $C = 0$ and are fitting a line of the form $0 + Dt = b$. Given the values of $b_1$ and $b_2$ at $t_1 = 1$ and $t_2 = 2$ respectively we have the following system of two equations:

$\begin{array}{rcrcr} 0&+&D&=&1 \\ 0&+&2D&=&7 \end{array}$

This system corresponds to the general matrix equation

$A \begin{bmatrix} C \\ D \end{bmatrix} = b$

which in this case becomes

$\begin{bmatrix} 0&1 \\ 0&2 \end{bmatrix} \begin{bmatrix} 0 \\ D \end{bmatrix} = \begin{bmatrix} 1 \\ 7 \end{bmatrix}$

This system has no solution. However we can find a least squares solution in general using the matrix equation

$A^TA \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = A^Tb$

In this case this corresponds to the equation

$\begin{bmatrix} 0&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} 0&1 \\ 0&2 \end{bmatrix} \begin{bmatrix} 0 \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 0&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} 1 \\ 7 \end{bmatrix}$

or

$\begin{bmatrix} 0&0 \\ 0&5 \end{bmatrix} \begin{bmatrix} 0 \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 0 \\ 15 \end{bmatrix}$

We then have $5 \bar{D} = 15$ or $\bar{D} = 3$.

So the least squares solution is the line $b = 3t$. This line passes through the origin (i.e., at $t = 0$ it has a value of zero). At time $t_1 = 1$ it has the value 3, which is greater than the value $b_1 = 1$. At time $t_2 = 2$ it has the value 6, which is less than the value $b_2 = 7$.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books.

This entry was posted in linear algebra and tagged , . Bookmark the permalink.