Exercise 3.3.1. a) Given the system of equations consisting of and find the least squares solution and describe the error being minimized by that solution. Confirm that the error vector is orthogonal to the column .

Answer: This is a system where and . We can then compute the least squares solution as .

We have

so that . We then have

So the least squares solution is .

The error vector is then

and the corresponding error

The inner product of the error vector and the column of is . Since the inner product is zero the two vectors are orthogonal.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fourth Edition and the accompanying free online course, and Dr Strang’s other books.