Linear Algebra and Its Applications, Exercise 3.3.2

Exercise 3.3.2. We have the value b_1 = 1 at time t_1 = 1 and the value b_2 = 7 at time t_2 = 2. We wish to fit these values using a line constrained to go through the origin, i.e., with an equation of the form b = Dt. Solve the system using least squares and describe the best fit line.

Answer: In general we would fit a line of the form C + Dt = b; however since we want the fitted line to go through the origin we have C = 0 and are fitting a line of the form 0 + Dt = b. Given the values of b_1 and b_2 at t_1 = 1 and t_2 = 2 respectively we have the following system of two equations:

\begin{array}{rcrcr} 0&+&D&=&1 \\ 0&+&2D&=&7 \end{array}

This system corresponds to the general matrix equation

A \begin{bmatrix} C \\ D \end{bmatrix} = b

which in this case becomes

\begin{bmatrix} 0&1 \\ 0&2 \end{bmatrix} \begin{bmatrix} 0 \\ D \end{bmatrix} = \begin{bmatrix} 1 \\ 7 \end{bmatrix}

This system has no solution. However we can find a least squares solution in general using the matrix equation

A^TA \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = A^Tb

In this case this corresponds to the equation

\begin{bmatrix} 0&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} 0&1 \\ 0&2 \end{bmatrix} \begin{bmatrix} 0 \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 0&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} 1 \\ 7 \end{bmatrix}


\begin{bmatrix} 0&0 \\ 0&5 \end{bmatrix} \begin{bmatrix} 0 \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 0 \\ 15 \end{bmatrix}

We then have 5 \bar{D} = 15 or \bar{D} = 3.

So the least squares solution is the line b = 3t. This line passes through the origin (i.e., at t = 0 it has a value of zero). At time t_1 = 1 it has the value 3, which is greater than the value b_1 = 1. At time t_2 = 2 it has the value 6, which is less than the value b_2 = 7.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books.

 Buy me a snack to sponsor more posts like this!

This entry was posted in linear algebra and tagged , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s