My Math Homework

Linear Algebra and Its Applications, Exercise 3.3.22

Posted on December 17, 2016 by hecker

Exercise 3.3.22. Given measurements of $b = 4, 2, -1, 0, 0$ at $t = -2, -1, 0, 1, 2$ use least squares to find the line of best fit of the form $C + Dt$ .

Answer: This corresponds to a system $Ax = b$ as follows:

$\begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&1 \\ 1&2 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} 4 \\ 2 \\ -1 \\ 0 \\ 0 \end{bmatrix}$

To find the least squares solution we form the system $A^TA\bar{x} = A^Tb$ . We have

$A^TA = \begin{bmatrix} 1&1&1&1&1 \\ -2&-1&0&1&2 \end{bmatrix} \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&1 \\ 1&2 \end{bmatrix}$

$= \begin{bmatrix} 5&0 \\ 0&10 \end{bmatrix}$

and

$A^Tb = \begin{bmatrix} 1&1&1&1&1 \\ -2&-1&0&1&2 \end{bmatrix} \begin{bmatrix} 4 \\ 2 \\ -1 \\ 0 \\ 0 \end{bmatrix} = \begin{bmatrix} 5 \\ -10 \end{bmatrix}$

The system $A^TA\bar{x} = A^Tb$ is then

$\begin{bmatrix} 5&0 \\ 0&10 \end{bmatrix} \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 5 \\ -10 \end{bmatrix}$

From the second equation we have $\bar{D} = -1$ and from the first equation we have $\bar{C} = 1$ . The line of best fit is therefore $1 - t$ .

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged least squares | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.21

Posted on December 10, 2016 by hecker

Exercise 3.3.21. Given three vectors $a_1$ , $a_2$ , and $b$ , and the two lines $L_1$ through the origin and $a_1$ and $L_2$ through $b$ in the direction of $a_2$ , we want to find scalar values $x_1$ and $x_2$ such that the distance $\| x_1a_1 - x_2a_2 - b \|$ between the points $x_1a_1$ and $b + x_2a_2$ is at a minimum. Write down equations for $x_1$ and $x_2$ . Solve the equations for $x = (x_1, x_2)$ when $a_1 = (1, 1, 0)$ , $a_2 = (0, 1, 0)$ , and $b = (2, 1, 4)$ .

Answer: We approach the problem by taking partial derivatives of $\| x_1a_1 - x_2a_2 - b \|^2$ with respect to $x_1$ and $x_2$ and then setting those partial derivatives to zero. We have

$\| x_1a_1 - x_2a_2 - b \|^2 = (x_1a_1 - x_2a_2 - b)^T(x_1a_1 - x_2a_2 - b)$

We take advantage of the facts that for two vectors $v$ and $w$ we have $v^Tw = w^Tv$ (the inner product is the same no matter in which order we calculate it) and $(v-w)^T = v^T - w^T$ (the transpose of a difference is the same as the difference of the transposes). We then have

$\| x_1a_1 - x_2a_2 - b \|^2 = (x_1a_1^T - x_2a_2^T - b^T)(x_1a_1 - x_2a_2 - b)$

$= x_1^2a_1^Ta_1 - x_1x_2a_1^Ta_2 - x_1a_1^Tb - x_2x_1a_2^Ta_1 + x_2^2a_2^Ta_2$

$+ x_2a_2^Tb - x_1b^Ta_1 + x_2b^Ta_2 + b^Tb$

$= x_1^2a_1^Ta_1 + x_2^2a_2^Ta_2 - 2x_1x_2a_1^Ta_2 - 2x_1a_1^Tb + 2x_2a_2^Tb + b^Tb$

Next we take the partial derivatives with respect to $x_1$ :

$\frac{\partial}{\partial x_1} \| x_1a_1 - x_2a_2 - b \|^2$

$= \frac{\partial}{\partial x_1} (x_1^2a_1^Ta_1 + x_2^2a_2^Ta_2 - 2x_1x_2a_1^Ta_2 - 2x_1a_1^Tb + 2x_2a_2^Tb + b^Tb)$

$= 2x_1a_1^Ta_1 - 2x_2a_1^Ta_2 - 2a_1^Tb$

and with respect to $x_2$ :

$\frac{\partial}{\partial x_2} \| x_1a_1 - x_2a_2 - b \|^2$

$= \frac{\partial}{\partial x_2} (x_1^2a_1^Ta_1 + x_2^2a_2^Ta_2 - 2x_1x_2a_1^Ta_2 - 2x_1a_1^Tb + 2x_2a_2^Tb + b^Tb)$

$= 2x_2a_2^Ta_2 - 2x_1a_1^Ta_2 + 2a_2^Tb$

Equating the partial derivatives to zero gives us the following system of equations (after simplifying by dividing by 2 and rearranging terms):

$\begin{bmatrix} a_1^Ta_1&-a_1^Ta_2 \\ -a_1^Ta_2&a_2^Ta_2 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} a_1^Tb \\ -a_2^Tb \end{bmatrix}$

When $a_1 = (1, 1, 0)$ , $a_2 = (0, 1, 0)$ , and $b = (2, 1, 4)$ we have $a_1^Ta_1 = 2$ , $a_2^Ta_2 = 1$ , $a_1^Ta_2 = 1$ , $a_1^Tb = 3$ , and $a_2^Tb = 1$ . The system of equations is then

$\begin{bmatrix} 2&-1 \\ -1&1 \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 3 \\ -1 \end{bmatrix}$

We can solve this by multiplying both sides of the equation on the left by

$\begin{bmatrix} 2&-1 \\ -1&1 \end{bmatrix}^{-1} = \frac{1}{2 - (-1)(-1)} \begin{bmatrix} 1&1 \\ 1&2 \end{bmatrix} = \begin{bmatrix} 1&1 \\ 1&2 \end{bmatrix}$

to get

$\begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} 1&1 \\ 1&2 \end{bmatrix} \begin{bmatrix} 3 \\ -1 \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}$

The solution is thus $x = (2, 1)$ .

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged least squares | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.20

Posted on December 3, 2016 by hecker

Exercise 3.3.20. Given the matrix $P_R$ that projects onto the row space of $A$ , find the matrix $P_N$ that projects onto the nullspace of $A$ .

Answer: The null space of $A$ is orthogonal to the row space of $A$ . The two spaces are orthogonal complements, with $\mathcal{N}(A) = \mathcal{R}(A^T)^\perp$ . Recall from exercise 3.3.11 that if $P$ is a projection matrix onto $S$ and $Q$ a projection matrix onto $S^\perp$ then we have $P+Q=I$ .

So in this case we have $P_R + P_N = I$ or $P_N = I - P_R$ .

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged nullspace, projection matrix, row space | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.19

Posted on November 26, 2016 by hecker

Exercise 3.3.19. Given a matrix $A$ , the matrix $P_C = A(A^TA)^{-1}A^T$ projects onto the column space of $A$ . Find the matrix $P_R$ that projects onto the row space of $A$ .

Answer: The row space of $A$ is the column space of $A^T$ . We can then find the matrix that projects onto $\mathcal{R}(A^T)$ using the standard formula but substituting $A^T$ for $A$ :

$P_R = A^T[(A^T)^TA^T]^{-1}(A^T)^T = A^T(AA^T)^{-1}A$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged projection matrix, row space | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.18

Posted on November 19, 2016 by hecker

Exercise 3.3.18. Suppose we have the following measurements of $y$ :

$y = 3, t = 1, z = 1$

$y = 5, t = 2, z = 1$

$y = 6, t = 0, z = 3$

$y = 0, t = 0, z = 0$

and want to fit a plane of the form $y = C + Dt + Ez$ .

a) Write a system of 4 equations in 3 unknowns representing the problem. (The system may not have a solution.)

b) Write a system of 3 equations in 3 unknowns representing the least squares solution to the problem.

Answer: a) The original problem can be expressed as the following system $Ax = b$ of 4 equations in 3 unknowns:

$\begin{bmatrix} 1&1&1 \\ 1&2&1 \\ 1&0&3 \\ 1&0&0 \end{bmatrix} \begin{bmatrix} C \\ D \\ E \end{bmatrix} = \begin{bmatrix} 3 \\ 5 \\ 6 \\ 0 \end{bmatrix}$

From the fourth equation we have $C=0$ and then substituting into the third equation we have $E=2$ . However we then get differing values of $D$ depending on whether we substitute into the second equation or the first. The system has no solution $x$ as written.

b) The least square solutions amounts to solving the system $A^TA\bar{x} = A^Tb$ that minimizes the error vector $b - A\bar{x}$ . We have

$A^TA = \begin{bmatrix} 1&1&1&1 \\ 1&2&0&0 \\ 1&1&3&0 \end{bmatrix} \begin{bmatrix} 1&1&1 \\ 1&2&1 \\ 1&0&3 \\ 1&0&0 \end{bmatrix}$

$= \begin{bmatrix} 4&3&5 \\ 3&5&3 \\ 5&3&11 \end{bmatrix}$

and

$A^Tb = \begin{bmatrix} 1&1&1&1 \\ 1&2&0&0 \\ 1&1&3&0 \end{bmatrix} \begin{bmatrix} 3 \\ 5 \\ 6 \\ 0 \end{bmatrix} = \begin{bmatrix} 14 \\ 13 \\ 26 \end{bmatrix}$

so that the resulting system of 3 equations in 3 unknowns $A^TA\bar{x} = A^Tb$ is

$\begin{bmatrix} 4&3&5 \\ 3&5&3 \\ 5&3&11 \end{bmatrix} \begin{bmatrix} \bar{C} \\ \bar{D} \\ \bar{E} \end{bmatrix} = \begin{bmatrix} 14 \\ 13 \\ 26 \end{bmatrix}$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged least squares, plane | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.17

Posted on November 12, 2016 by hecker

Exercise 3.3.17. Find the projection matrix $P$ that projects vectors in $\mathbb{R}^2$ onto the line $x+y=0$ .

Answer: The vector $(1, -1)$ is a basis for the subspace being projected onto, which is thus the column space of

$A = \begin{bmatrix} 1 \\ -1 \end{bmatrix}$

Using the formula $P = A(A^TA)^{-1}AT$ we have

$A^TA = \begin{bmatrix} 1&-1 \end{bmatrix} \begin{bmatrix} 1 \\ -1 \end{bmatrix} = 2$

so that $(A^TA)^{-1} = \frac{1}{2}$ and

$P = A(A^TA)^{-1}AT = \begin{bmatrix} 1 \\ -1 \end{bmatrix} \cdot \frac{1}{2} \cdot \begin{bmatrix} 1&-1 \end{bmatrix}$

$= \begin{bmatrix} \frac{1}{2}&-\frac{1}{2} \\ -\frac{1}{2}&\frac{1}{2} \end{bmatrix}$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged line, projection matrix | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.16

Posted on November 5, 2016 by hecker

Exercise 3.3.16. Suppose $u$ is a vector with unit length. Show that the matrix $uu^T$ (with rank 1) is a projection matrix.

Answer: We have

$(uu^T)^2 = (uu^T)(uu^T) = u(u^Tu)u^T$

But since $u$ has unit length we have $u^Tu = \|u\|^2 = 1$ so that

$(uu^T)^2 = u \cdot 1 \cdot u^T = uu^T$

We also have

$(uu^T)^T = (u^T)^Tu^T = uu^T$

Since $(uu^T)^2 = uu^T$ and $(uu^T)^T = uu^T$ the rank-1 matrix $uu^T$ is a projection matrix.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged projection matrix, unit vector | 1 Comment

Linear Algebra and Its Applications, Exercise 3.3.15

Posted on October 29, 2016 by hecker

Exercise 3.3.15. Suppose $P$ is a projection matrix that projects vectors onto a line in the $x$ – $y$ plane. Describe the effect of the reflection matrix $H = I-2P$ geometrically. Why does $H^2 = I$ ? (Give both a geometric and algebraic explanation.)

Answer: When applied to a vector $v$ the matrix $H$ produces $Hv = (I-2P)v = v - 2Pv$ . This can be thought of combining the following operations:

First, project $v$ onto the line in the $x$ – $y$ plane ( $Pv$ ). Next, scale the resulting vector by a factor of 2 and reverse its direction ( $-2Pv$ ). The resulting vector is still on the line onto which $P$ projects. Finally, find the difference between the resulting vector and the original vector $v$ ( $v - 2Pv$ ). The final vector $Hv$ and the original vector $v$ are symmetric with respect to a line that is perpendicular to the original line onto which $P$ projects.

For a simple example, in $\mathbb{R}^2$ consider the projection matrix

$P = \begin{bmatrix} 1&0 \\ 0&0 \end{bmatrix}$

that projects all vectors onto the $x$ axis.

We then have

$H = I -2P = \begin{bmatrix} 1&0 \\ 0&1 \end{bmatrix} - 2 \cdot \begin{bmatrix} 1&0 \\ 0&0 \end{bmatrix}$

$= \begin{bmatrix} 1&0 \\ 0&1 \end{bmatrix} - \begin{bmatrix} 2&0 \\ 0&0 \end{bmatrix} = \begin{bmatrix} -1&0 \\ 0&1 \end{bmatrix}$

When $H$ is applied to the vector $(2, 3)$ this produces the vector $(-2, 3)$ :

$\begin{bmatrix} -1&0 \\ 0&1 \end{bmatrix} \begin{bmatrix} 2 \\ 3 \end{bmatrix} = \begin{bmatrix} -2 \\ 3 \end{bmatrix}$

In other words, $(2, 3)$ is reflected about the $y$ axis, which is perpendicular to the $x$ axis onto which $P$ projects.

If $H = I - 2P$ is applied again then it would reflect the vector $(-2, 3)$ back across the $y$ axis to produce the original vector $(2, 3)$ . This is true in general for any $H = I-2P$ :

$H^2 = (I-2P)(I-2P) = I^2 - 2IP - 2PI + 4P^2$

$= I - 2P -2P + 4P = I$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged projection matrix, reflection matrix | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.14

Posted on October 22, 2016 by hecker

Exercise 3.3.14. Find the projection matrix $P$ onto the plane spanned by the vectors $(1, 1, 0)$ and $(1, 1, 1)$ . Find a nonzero vector $b$ that $P$ projects to zero.

Answer: The plane in question is the column space of the matrix

$A = \begin{bmatrix} 1&1 \\ 1&1 \\ 0&1 \end{bmatrix}$

The projection matrix $P = A(A^TA)^{-1}A^T$ . We have

$A^TA = \begin{bmatrix} 1&1&0 \\ 1&1&1 \end{bmatrix} \begin{bmatrix} 1&1 \\ 1&1 \\ 0&1 \end{bmatrix} = \begin{bmatrix} 2&2 \\ 2&3 \end{bmatrix}$

so that its inverse is

$(A^TA)^{-1} = \frac{1}{2 \cdot 3 - 2 \cdot 2} \begin{bmatrix} 3&-2 \\ -2&2 \end{bmatrix}$

$= \frac{1}{2} \begin{bmatrix} 3&-2 \\ -2&2 \end{bmatrix} = \begin{bmatrix} \frac{3}{2}&-1 \\ -1&1 \end{bmatrix}$

We then have

$P = A(A^TA)^{-1}A^T$

$= \begin{bmatrix} 1&1 \\ 1&1 \\ 0&1 \end{bmatrix} \begin{bmatrix} \frac{3}{2}&-1 \\ -1&1 \end{bmatrix} \begin{bmatrix} 1&1&0 \\ 1&1&1 \end{bmatrix}$

$= \begin{bmatrix} 1&1 \\ 1&1 \\ 0&1 \end{bmatrix} \begin{bmatrix} \frac{1}{2}&\frac{1}{2}&-1 \\ 0&0&1 \end{bmatrix}$

$= \begin{bmatrix} \frac{1}{2}&\frac{1}{2}&0 \\ \frac{1}{2}&\frac{1}{2}&0 \\ 0&0&1& \end{bmatrix}$

This completes the first part of the exercise. For the second part, we need to find a nonzero vector $b$ projected into zero by $P$ . This will be true if $b$ is in the left nullspace of $A$ , since $b$ will then be orthogonal to the plane corresponding to the column space of $A$ .

We therefore look for a vector in the left nullspace $\mathcal{N}(A^T)$ by solving the system $A^Ty = 0$ :

$\begin{bmatrix} 1&1&0 \\ 1&1&1 \end{bmatrix} \begin{bmatrix} y_1 \\ y_2 \\ y_3 \end{bmatrix} = 0$

We perform Gaussian elimination by multiplying the first row by 1 and subtracting it from the second row:

$\begin{bmatrix} 1&1&0 \\ 1&1&1 \end{bmatrix} \Rightarrow \begin{bmatrix} 1&1&0 \\ 0&0&1 \end{bmatrix}$

The resulting matrix is in echelon form, with pivots in columns 1 and 3. Thus $y_1$ and $y_3$ are basic variables, and $y_2$ is a free variable. Setting $y_2 = 1$ we have $y_3 = 0$ (from the second row) and $y_1 = -1$ (from the first row). The vector $b = (-1, 1, 0)$ is therefore a solution to $A^Ty = 0$ and is a basis for the left nullspace of $A$ .

Since $b$ is in $\mathcal{N}(A^T)$ it is orthogonal to $\mathcal{R}(A)$ and is therefore projected by $P$ into zero:

$Pb = \begin{bmatrix} \frac{1}{2}&\frac{1}{2}&0 \\ \frac{1}{2}&\frac{1}{2}&0 \\ 0&0&1& \end{bmatrix} \begin{bmatrix} -1 \\ 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged column space, left nullspace, plane, projection matrix | Leave a comment

Linear Algebra and Its Applications, Exercise 3.3.13

Posted on October 15, 2016 by hecker

Exercise 3.3.13. Using least squares, find the line that is the best fit to the following measurements:

$b = 4$ at $t = -2$

$b = 3$ at $t = -1$

$b = 1$ at $t = 0$

$b = 0$ at $= 2$

Also, given the matrix

$A = \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&2 \end{bmatrix}$

find the projection of $b = (4, 3, 1, 0)$ onto the column space $\mathcal{R}(A)$ .

Answer: Assuming that the line in question has the form $C + Dt$ the problem can be expressed as that of finding a solution to the system

$\begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} 4 \\ 3 \\ 1 \\ 0 \end{bmatrix}$

or $Ax = b$ where $x = (C, D)$ is the exact solution.

In this case there is no exact solution, so we look for the least squares solution $\bar{x} = (\bar{C}, \bar{D})$ that minimizes the error vector $b-A\bar{x}$ . The error vector is minimized when it is orthogonal to the column space of $A$ and is therefore in the left nullspace of $A$ . We then have $A^T(b - A\bar{x}) = 0$ so that $\bar{x}$ is a solution to the system $A^TA\bar{x} = A^Tb$ .

We have

$A^TA = \begin{bmatrix} 1&1&1&1 \\ -2&-1&0&2 \end{bmatrix} \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&2 \end{bmatrix} = \begin{bmatrix} 4&-1 \\ -1&9 \end{bmatrix}$

and

$A^Tb = \begin{bmatrix} 1&1&1&1 \\ -2&-1&0&2 \end{bmatrix} \begin{bmatrix} 4 \\ 3 \\ 1 \\ 0 \end{bmatrix} = \begin{bmatrix} 8 \\ -11 \end{bmatrix}$

so that the system $A^TA\bar{x} = A^Tb$ reduces to

$\begin{bmatrix} 4&-1 \\ -1&9 \end{bmatrix} \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = \begin{bmatrix} 8 \\ -11 \end{bmatrix}$

$\setlength\arraycolsep{0.2em}\begin{array}{rcrcl}4\bar{C}&-&\bar{D}&=&8 \\ -\bar{C}&+&9\bar{D}&=&-11 \end{array}$

expressed as a system of equations.

Multiplying the first equation by $\frac{1}{4}$ and adding it to the second equation produces the system

$\setlength\arraycolsep{0.2em}\begin{array}{rcrcl}4\bar{C}&-&\bar{D}&=&8 \\ &&\frac{35}{4}\bar{D}&=&-9 \end{array}$

From the second equation we have $\bar{D} = -\frac{36}{35}$ . Substituting that value into the first equation we have $4\bar{C} + \frac{36}{35} = 8$ or $\bar{C} = \frac{1}{4}(8 - \frac{36}{35}) = \frac{61}{35}$ .

The line of best fit is therefore $\frac{61}{35} - \frac{36}{35}t$ .

Given the matrix

$A = \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&2 \end{bmatrix}$

the projection matrix $P$ onto the column space of $A$ can be computed as

$P = A(A^TA)^{-1}A^T$

From above we have

$A^TA = \begin{bmatrix} 4&-1 \\ -1&9 \end{bmatrix}$

so that its inverse is

$(A^TA)^{-1} = \frac{1}{4 \cdot 9 - (-1)(-1)} \begin{bmatrix} 9&-(-1) \\ -(-1)&4 \end{bmatrix}$

$= \frac{1}{35} \begin{bmatrix} 9&1 \\ 1&4 \end{bmatrix}$

We then have

$P = A(A^TA)^{-1}A^T$

$= \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&2 \end{bmatrix} \frac{1}{35} \begin{bmatrix} 9&1 \\ 1&4 \end{bmatrix} \begin{bmatrix} 1&1&1&1 \\ -2&-1&0&2 \end{bmatrix}$

$= \frac{1}{35} \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&0 \\ 1&2 \end{bmatrix} \begin{bmatrix} 7&8&9&11 \\ -7&-3&1&9 \end{bmatrix}$

$= \frac{1}{35} \begin{bmatrix} 21&14&7&-7 \\ 14&11&8&2 \\ 7&8&9&11 \\ -7&2&11&29 \end{bmatrix}$

The projection of the vector $b = (4, 3, 1, 0)$ onto the column space of $A$ is then

$Pb = \frac{1}{35} \begin{bmatrix} 21&14&7&-7 \\ 14&11&8&2 \\ 7&8&9&11 \\ -7&2&11&29 \end{bmatrix} \begin{bmatrix} 4 \\ 3 \\ 1 \\ 0 \end{bmatrix}$

$= \frac{1}{35} \begin{bmatrix} 133 \\ 97 \\ 61 \\ -11 \end{bmatrix} = \begin{bmatrix} \frac{133}{35} \\ \frac{97}{35} \\ \frac{61}{35} \\ -\frac{11}{35} \end{bmatrix}$

The vector $Pb$ corresponds to the points on the least squares line of best fit $\bar{C} + \bar{D}t$ for the times $t = (-2, 1, 0, 2)$ :

$\frac{61}{35} - \frac{36}{35} (-2) = \frac{133}{35}$

$\frac{61}{35} - \frac{36}{35} (-1) = \frac{97}{35}$

$\frac{61}{35} - \frac{36}{35} (0) = \frac{61}{35}$

$\frac{61}{35} - \frac{36}{35} (2) = \frac{-11}{35}$

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

Posted in linear algebra | Tagged least squares, projection matrix | Leave a comment

My Math Homework

Linear Algebra and Its Applications, Exercise 3.3.22

Linear Algebra and Its Applications, Exercise 3.3.21

Linear Algebra and Its Applications, Exercise 3.3.20

Linear Algebra and Its Applications, Exercise 3.3.19

Linear Algebra and Its Applications, Exercise 3.3.18

Linear Algebra and Its Applications, Exercise 3.3.17

Linear Algebra and Its Applications, Exercise 3.3.16

Linear Algebra and Its Applications, Exercise 3.3.15

Linear Algebra and Its Applications, Exercise 3.3.14

Linear Algebra and Its Applications, Exercise 3.3.13

Archives

Meta