## Linear Algebra and Its Applications, Exercise 3.4.25

Exercise 3.4.25. Given $y = x^2$ over the interval $-1 \le x \le 1$ what is the closest line $C + Dx$ to the parabola formed by $y$?

Answer: This amounts to finding a least-squares solution to the equation $\begin{bmatrix} 1&x \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = y$, where the entries 1, $x$, and $y = x^2$ are understood as functions of $x$ over the interval -1 to 1 (as opposed to being scalar values).

Interpreting the traditional least squares equation $A^TAx = A^Tb$ in this context, here the matrix $A = \begin{bmatrix} 1&x \end{bmatrix}$ and we have $A^TA = \begin{bmatrix} 1 \\ x \end{bmatrix} \begin{bmatrix} 1&x \end{bmatrix} = \begin{bmatrix} (1, 1)&(1, x) \\ (x, 1)&(x, x) \end{bmatrix}$

where the entries of $A^TA$ are the dot products of the functions, i.e., the integrals of their products over the interval -1 to 1.

We then have $(1, 1) = \int_{-1}^1 1 \cdot 1 \;\mathrm{d}x = 2$ $(1, x) = (x, 1) = \int_{-1}^1 1 \cdot x \;\mathrm{d}x = \left( \frac{1}{2}x^2 \right) \;\big|_{-1}^1 = \frac{1}{2} \cdot 1^2 - \frac{1}{2} \cdot (-1)^2 = \frac{1}{2} - \frac{1}{2} = 0$ $(x, x) = \int_{-1}^1 x^2 \;\mathrm{d}x = \left( \frac{1}{3}x^3 \right) \;\big|_{-1}^1 = \frac{1}{3} \cdot 1^3 - \frac{1}{3} \cdot (-1)^3 = \frac{1}{3} + \frac{1}{3} = \frac{2}{3}$

so that $A^TA = \begin{bmatrix} (1, 1)&(1, x) \\ (x, 1)&(x, x) \end{bmatrix} = \begin{bmatrix} 2&0 \\ 0&\frac{2}{3} \end{bmatrix}$

Continuing the interpretation of the least squares equation $A^TAx = A^Tb$ in this context, the role of $b$ is played by the function $y = x^2$, and we have $A^Ty = \begin{bmatrix} 1 \\ x \end{bmatrix} x^2 = \begin{bmatrix} (1,x^2) \\ (x, x^2) \end{bmatrix}$

where again the entries are dot products of the functions. From above we have $(1, x^2) = \int_{-1}^1 1 \cdot x^2 \;\mathrm{d}x = \frac{2}{3}$

and from previous exercises we have $(x, x^2) = \int_{-1}^1 x \cdot x^2 \;\mathrm{d}x = \int_{-1}^1 x^3 \;\mathrm{d}x = 0$

so that $A^Ty = \begin{bmatrix} \frac{2}{3} \\ 0 \end{bmatrix}$

To get the least squares solution $\bar{C} + \bar{D}x$ we then have $\begin{bmatrix} 2&0 \\ 0&\frac{2}{3} \end{bmatrix} \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = \begin{bmatrix} \frac{2}{3} \\ 0 \end{bmatrix}$

From the second equation we have $\bar{D} = 0$. From the first equation we have $2\bar{C} = \frac{2}{3}$ or $C = \frac{1}{3}$.

The line of best fit to the parabola $y = x^2$ over the interval $-1 \le x \le 1$ is therefore the horizontal line with $y$-intercept of $\frac{1}{3}$.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged , . Bookmark the permalink.