## Linear Algebra and Its Applications, Exercise 3.3.2

Exercise 3.3.2. We have the value $b_1 = 1$ at time $t_1 = 1$ and the value $b_2 = 7$ at time $t_2 = 2$. We wish to fit these values using a line constrained to go through the origin, i.e., with an equation of the form $b = Dt$. Solve the system using least squares and describe the best fit line.

Answer: In general we would fit a line of the form $C + Dt = b$; however since we want the fitted line to go through the origin we have $C = 0$ and are fitting a line of the form $0 + Dt = b$. Given the values of $b_1$ and $b_2$ at $t_1 = 1$ and $t_2 = 2$ respectively we have the following system of two equations: $\begin{array}{rcrcr} 0&+&D&=&1 \\ 0&+&2D&=&7 \end{array}$

This system corresponds to the general matrix equation $A \begin{bmatrix} C \\ D \end{bmatrix} = b$

which in this case becomes $\begin{bmatrix} 0&1 \\ 0&2 \end{bmatrix} \begin{bmatrix} 0 \\ D \end{bmatrix} = \begin{bmatrix} 1 \\ 7 \end{bmatrix}$

This system has no solution. However we can find a least squares solution in general using the matrix equation $A^TA \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = A^Tb$

In this case this corresponds to the equation $\begin{bmatrix} 0&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} 0&1 \\ 0&2 \end{bmatrix} \begin{bmatrix} 0 \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 0&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} 1 \\ 7 \end{bmatrix}$

or $\begin{bmatrix} 0&0 \\ 0&5 \end{bmatrix} \begin{bmatrix} 0 \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 0 \\ 15 \end{bmatrix}$

We then have $5 \bar{D} = 15$ or $\bar{D} = 3$.

So the least squares solution is the line $b = 3t$. This line passes through the origin (i.e., at $t = 0$ it has a value of zero). At time $t_1 = 1$ it has the value 3, which is greater than the value $b_1 = 1$. At time $t_2 = 2$ it has the value 6, which is less than the value $b_2 = 7$.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , . Bookmark the permalink.