## Linear Algebra and Its Applications, Exercise 3.3.21

Exercise 3.3.21. Given three vectors $a_1$, $a_2$, and $b$, and the two lines $L_1$ through the origin and $a_1$ and $L_2$ through $b$ in the direction of $a_2$, we want to find scalar values $x_1$ and $x_2$ such that the distance $\| x_1a_1 - x_2a_2 - b \|$ between the points $x_1a_1$ and $b + x_2a_2$ is at a minimum. Write down equations for $x_1$ and $x_2$. Solve the equations for $x = (x_1, x_2)$ when $a_1 = (1, 1, 0)$, $a_2 = (0, 1, 0)$, and $b = (2, 1, 4)$.

Answer: We approach the problem by taking partial derivatives of $\| x_1a_1 - x_2a_2 - b \|^2$ with respect to $x_1$ and $x_2$ and then setting those partial derivatives to zero. We have $\| x_1a_1 - x_2a_2 - b \|^2 = (x_1a_1 - x_2a_2 - b)^T(x_1a_1 - x_2a_2 - b)$

We take advantage of the facts that for two vectors $v$ and $w$ we have $v^Tw = w^Tv$ (the inner product is the same no matter in which order we calculate it) and $(v-w)^T = v^T - w^T$ (the transpose of a difference is the same as the difference of the transposes). We then have $\| x_1a_1 - x_2a_2 - b \|^2 = (x_1a_1^T - x_2a_2^T - b^T)(x_1a_1 - x_2a_2 - b)$ $= x_1^2a_1^Ta_1 - x_1x_2a_1^Ta_2 - x_1a_1^Tb - x_2x_1a_2^Ta_1 + x_2^2a_2^Ta_2$ $+ x_2a_2^Tb - x_1b^Ta_1 + x_2b^Ta_2 + b^Tb$ $= x_1^2a_1^Ta_1 + x_2^2a_2^Ta_2 - 2x_1x_2a_1^Ta_2 - 2x_1a_1^Tb + 2x_2a_2^Tb + b^Tb$

Next we take the partial derivatives with respect to $x_1$: $\frac{\partial}{\partial x_1} \| x_1a_1 - x_2a_2 - b \|^2$ $= \frac{\partial}{\partial x_1} (x_1^2a_1^Ta_1 + x_2^2a_2^Ta_2 - 2x_1x_2a_1^Ta_2 - 2x_1a_1^Tb + 2x_2a_2^Tb + b^Tb)$ $= 2x_1a_1^Ta_1 - 2x_2a_1^Ta_2 - 2a_1^Tb$

and with respect to $x_2$: $\frac{\partial}{\partial x_2} \| x_1a_1 - x_2a_2 - b \|^2$ $= \frac{\partial}{\partial x_2} (x_1^2a_1^Ta_1 + x_2^2a_2^Ta_2 - 2x_1x_2a_1^Ta_2 - 2x_1a_1^Tb + 2x_2a_2^Tb + b^Tb)$ $= 2x_2a_2^Ta_2 - 2x_1a_1^Ta_2 + 2a_2^Tb$

Equating the partial derivatives to zero gives us the following system of equations (after simplifying by dividing by 2 and rearranging terms): $\begin{bmatrix} a_1^Ta_1&-a_1^Ta_2 \\ -a_1^Ta_2&a_2^Ta_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} a_1^Tb \\ -a_2^Tb \end{bmatrix}$

When $a_1 = (1, 1, 0)$, $a_2 = (0, 1, 0)$, and $b = (2, 1, 4)$ we have $a_1^Ta_1 = 2$, $a_2^Ta_2 = 1$, $a_1^Ta_2 = 1$, $a_1^Tb = 3$, and $a_2^Tb = 1$. The system of equations is then $\begin{bmatrix} 2&-1 \\ -1&1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 3 \\ -1 \end{bmatrix}$

We can solve this by multiplying both sides of the equation on the left by $\begin{bmatrix} 2&-1 \\ -1&1 \end{bmatrix}^{-1} = \frac{1}{2 - (-1)(-1)} \begin{bmatrix} 1&1 \\ 1&2 \end{bmatrix} = \begin{bmatrix} 1&1 \\ 1&2 \end{bmatrix}$

to get $\begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 1&1 \\ 1&2 \end{bmatrix} \begin{bmatrix} 3 \\ -1 \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}$

The solution is thus $x = (2, 1)$.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition , Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books .

This entry was posted in linear algebra and tagged . Bookmark the permalink.