Linear Algebra and Its Applications, Exercise 3.4.10

Exercise 3.4.10. Given the two orthonormal vectors q_1 and q_2 and an arbitrary vector b, what linear combination of q_1 and q_2 is the least distance from b? Show that the difference between b and that combination (i.e., the error vector) is orthogonal to both q_1 and q_2.

Answer: This exercise is similar to the previous one. Any linear combination of q_1 and q_2 is in the plane formed by q_1 and q_2, and the combination closest to b is the projection p of b onto that plane. Because q_1 and q_2 are orthonormal that projection is equal to the sum of the separate projections of b onto q_1 and q_2 respectively:

p = (q_1^Tb)q_1 + (q_2^Tb)q_2

So (q_1^Tb)q_1 + (q_2^Tb)q_2 is the closest combination to b.

The error vector is then

e = b - p = b - (q_1^Tbq_1 + q_2^Tbq_2)

Taking the dot product of the error vector with q_1 we have

q_1^Te = q_1^T [b - (q_1^Tbq_1 + q_2^Tbq_2)] = q_1^Tb - q_1^T(q_1^Tb)q_1 - q_1^T(q_2^Tb)q_2

= q_1^Tb - (q_1^Tb)q_1^Tq_1 - (q_2^Tb)q_1^Tq_2

Since q_1 and q_2 are orthonormal we have q_1^Tq_1 = 1 and q_1^Tq_2 = 0. So we have

q_1^Te = q_1^Tb - (q_1^Tb) \cdot 1 - (q_2^Tb) \cdot 0 = q_1^Tb - q_1^Tb = 0

Similarly we have

q_2^Te = q_2^T [b - (q_1^Tbq_1 + q_2^Tbq_2)] = q_2^Tb - q_2^T(q_1^Tb)q_1 - q_2^T(q_2^Tb)q_2

= q_2^Tb - (q_1^Tb)q_2^Tq_1 - (q_2^Tb)q_2^Tq_2 = q_2^Tb - q_2^Tb = 0

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged , | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.9

Exercise 3.4.9. Given the three orthonormal vectors q_1, q_2, and q_3, what linear combination of q_1 and q_2 is the least distance from q_3?

Answer: Any linear combination of q_1 and q_2 is in the plane formed by q_1 and q_2. The combination closest to q_3 is simply the projection of q_3 onto that plane. But because q_3 is orthogonal to both q_1 and q_2 it is orthogonal to that plane, and its projection onto the plane is the zero vector. So the linear combination of q_1 and q_2 closest to q_3 is 0 \cdot q_1 + 0 \cdot q_2.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged , | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.8

Exercise 3.4.8. Project the vector b = (1, 2) onto the two non-orthogonal vectors a_1 = (1, 0) and a_2 = (1, 1) and show that the sum of the two projections does not equal b (as it would if a_1 and a_2 were orthogonal).

Answer: The projection of b onto a_1 is (a_1^Tb/a_1^Ta_1)a_1. We have a_1^Tb = (1 \cdot 1 + 0 \cdot 2) = 1 and a_1^Ta_1 = (1 \cdot 1 + 0 \cdot 0) = 1. So the projection of b onto a_1 is \frac{1}{1} a_1 = (1, 0).

Similarly, the projection of b onto a_2 is (a_2^Tb/a_2^Ta_2)a_2. We have a_2^Tb = (1 \cdot 1 + 1 \cdot 2) = 3 and a_2^Ta_2 = (1 \cdot 1 + 1 \cdot 1) = 2. So the projection of b onto a_2 is \frac{3}{2} a_2 = (\frac{3}{2}, \frac{3}{2}).

The sum of the two projections is (1, 0) + (\frac{3}{2}, \frac{3}{2}) = (\frac{5}{2}, \frac{3}{2}), which is not equal to b.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.7

Exercise 3.4.7. Given b = x_1q_1 + x_2q_2 + \cdots + x_nq_n where q_1, q_2, \dots, q_n are orthonormal vectors, compute b^Tb and show that

\|b\|^2 = x_1^2 + x_2^2 + \cdots + x_n^2

Answer:We have b = \sum_i x_iq_i so that

b^Tb = (\sum_i x_iq_i)^T(\sum_j x_jq_j) = (\sum_i x_iq_i^T)(\sum_j x_jq_j)

since the transpose of a sum is equal to the sum of the transposes. The product of the sums can then be decomposed into two sums of products as follows:

(\sum_i x_iq_i^T)(\sum_j x_jq_j) = \sum_{i=j} (x_iq_i^T)(x_jq_j) + \sum_{i \ne j} (x_iq_i^T)(x_jq_j)

= \sum_{i=j} x_ix_jq_i^Tq_j + \sum_{i \ne j} x_ix_jq_i^Tq_j

But since q_1, q_2, \dots, q_n are orthonormal vectors we have q_i^Tq_j = 1 when i = j and q_i^Tq_j = 0 when i \ne j, so that

\sum_{i=j} x_ix_jq_i^Tq_j + \sum_{i \ne j} x_ix_jq_i^Tq_j = \sum_{i = j} x_ix_j \cdot 1 + \sum_{i \ne j} x_ix_j \cdot 0 = \sum_i x_i^2

We thus have

\|b\|^2 = \sum_i x_i^2 = x_1^2 + x_2^2 + \cdots + x_n^2

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.6

Exercise 3.4.6. Given the matrix

Q = \begin{bmatrix} \frac{1}{\sqrt{3}}&\frac{1}{\sqrt{14}}&\qquad \\ \frac{1}{\sqrt{3}}&\frac{2}{\sqrt{14}}&\qquad \\ \frac{1}{\sqrt{3}}&-\frac{3}{\sqrt{14}}&\qquad \end{bmatrix}

find entries for the third column such that Q is orthogonal. How much freedom do you have to choose the entries? Finally, verify that both the columns and rows are orthonormal.

Answer: In order for Q to be orthogonal all three of its rows must be orthonormal, with length of 1. We start with the vector forming the first row of Q.  For its length to be 1 the square of the last entry of the first row must be equal to

1 - (\frac{1}{\sqrt{3}})^2 - (\frac{1}{\sqrt{14}})^2 = 1 - \frac{1}{3} - \frac{1}{14} = 1 - \frac{14}{42} - \frac{3}{42} = \frac{25}{42}

so that the entry itself would be \pm \sqrt{\frac{25}{42}} = \pm \frac{5}{\sqrt{42}}.

Similarly the square of the last entry of the second row must be

1 - (\frac{1}{\sqrt{3}})^2 - (\frac{2}{\sqrt{14}})^2 = 1 - \frac{1}{3} - \frac{4}{14} = 1 - \frac{14}{42} - \frac{12}{42} = \frac{16}{42}

so that the entry itself would be \pm \sqrt{\frac{16}{42}} = \pm \frac{4}{\sqrt{42}}.

Finally, the square of the last entry of the third row must be

1 - (\frac{1}{\sqrt{3}})^2 - (\frac{3}{\sqrt{14}})^2 = 1 - \frac{1}{3} - \frac{9}{14} = 1 - \frac{14}{42} - \frac{27}{42} = \frac{1}{42}

so that the entry itself would be \pm \sqrt{\frac{1}{42}} = \pm \frac{1}{\sqrt{42}}.

We have two possible choices for the last entry of the first row, \frac{5}{\sqrt{42}} and -\frac{5}{\sqrt{42}}. Suppose that we choose \frac{5}{\sqrt{42}}. Since the first row and the second row are supposed to be orthogonal, we must choose -\frac{4}{\sqrt{42}} for the last entry of the second row, so that the dot product of the two rows is zero:

\frac{1}{\sqrt{3}} \cdot \frac{1}{\sqrt{3}} + \frac{1}{\sqrt{14}} \cdot \frac{2}{\sqrt{14}} + \frac{5}{\sqrt{42}} \cdot (-\frac{4}{\sqrt{42}}) = \frac{1}{3} + \frac{2}{14} - \frac{20}{42} = \frac{14}{42} + \frac{6}{42} - \frac{20}{42} = 0

We must then choose -\frac{1}{\sqrt{42}} for the last entry in the third row, so that the dot product of the first row and the third row is zero:

\frac{1}{\sqrt{3}} \cdot \frac{1}{\sqrt{3}} + \frac{1}{\sqrt{14}} \cdot (-\frac{3}{\sqrt{14}}) + \frac{5}{\sqrt{42}} \cdot (-\frac{1}{\sqrt{42}}) = \frac{1}{3} - \frac{3}{14} - \frac{5}{42} = \frac{14}{42} - \frac{9}{42} - \frac{5}{42} = 0

and the dot product of the second and third rows is zero:

\frac{1}{\sqrt{3}} \cdot \frac{1}{\sqrt{3}} + \frac{2}{\sqrt{14}} \cdot (-\frac{3}{\sqrt{14}}) + (-\frac{4}{\sqrt{42}}) \cdot (-\frac{1}{\sqrt{42}}) = \frac{1}{3} - \frac{6}{14} + \frac{4}{42} = \frac{14}{42} - \frac{18}{42} + \frac{4}{42} = 0

The resulting value for the matrix Q is

Q = \begin{bmatrix} \frac{1}{\sqrt{3}}&\frac{1}{\sqrt{14}}&\frac{5}{\sqrt{42}} \\ \frac{1}{\sqrt{3}}&\frac{2}{\sqrt{14}}&-\frac{4}{\sqrt{42}} \\ \frac{1}{\sqrt{3}}&-\frac{3}{\sqrt{14}}&-\frac{1}{\sqrt{42}} \end{bmatrix}

The dot product of the first column with itself is

(\frac{1}{\sqrt{3}})^2 + (\frac{1}{\sqrt{3}})^2 + (\frac{1}{\sqrt{3}})^2 = \frac{1}{3} + \frac{1}{3} + \frac{1}{3} = 1

The dot product of the second column with itself is

(\frac{1}{\sqrt{14}})^2 + (\frac{2}{\sqrt{14}})^2 + (-\frac{3}{\sqrt{14}})^2 = \frac{1}{14} + \frac{4}{14} + \frac{9}{14} = 1

The dot product of the third column with itself is

(\frac{5}{\sqrt{42}})^2 + (-\frac{4}{\sqrt{42}})^2 + (-\frac{1}{\sqrt{42}})^2 = \frac{25}{42} + \frac{16}{42} + \frac{1}{42} = 1

Thus all three columns have length 1.

We also have the dot product of the first and second columns as zero:

\frac{1}{\sqrt{3}} \cdot \frac{1}{\sqrt{14}} + \frac{1}{\sqrt{3}} \cdot \frac{2}{\sqrt{14}} + \frac{1}{\sqrt{3}} \cdot (-\frac{3}{\sqrt{14}}) = \frac{1}{\sqrt{42}} + \frac{2}{\sqrt{42}} - \frac{3}{\sqrt{42}} = 0

the dot product of the first and third columns as zero:

\frac{1}{\sqrt{3}} \cdot \frac{5}{\sqrt{42}} + \frac{1}{\sqrt{3}} \cdot (-\frac{4}{\sqrt{42}}) + \frac{1}{\sqrt{3}} \cdot (-\frac{1}{\sqrt{42}}) = \frac{5}{\sqrt{126}} - \frac{4}{\sqrt{126}} - \frac{1}{\sqrt{126}} = 0

and the dot product of the second and third columns as zero:

\frac{1}{\sqrt{14}} \cdot \frac{5}{\sqrt{42}} + \frac{2}{\sqrt{14}} \cdot (-\frac{4}{\sqrt{42}}) + (-\frac{3}{\sqrt{14}}) \cdot (-\frac{1}{\sqrt{42}}) = \frac{5}{\sqrt{588}} - \frac{8}{\sqrt{588}} + \frac{3}{\sqrt{588}} = 0

Since all three rows are orthonormal (by construction) and all three columns are orthonormal, the matrix Q is an orthogonal matrix.

Recall that we originally had two choices for the last entry of the first row. If we instead choose -\frac{5}{\sqrt{42}} for the last entry in the first row, we must choose \frac{4}{\sqrt{42}} for the last entry of the second row and \frac{1}{\sqrt{42}} for the last entry in the third row, so that

Q = \begin{bmatrix} \frac{1}{\sqrt{3}}&\frac{1}{\sqrt{14}}&-\frac{5}{\sqrt{42}} \\ \frac{1}{\sqrt{3}}&\frac{2}{\sqrt{14}}&\frac{4}{\sqrt{42}} \\ \frac{1}{\sqrt{3}}&-\frac{3}{\sqrt{14}}&\frac{1}{\sqrt{42}} \end{bmatrix}

Verifying that this alternative value for Q is an orthogonal matrix is left as an exercise for the reader.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.5

Exercise 3.4.5. Given a unit vector u and Q = I - 2uu^T, prove that Q is orthogonal. What is Q when u = (\frac{1}{2},\frac{1}{2}, -\frac{1}{2}, -\frac{1}{2})?

Answer: We have

Q^TQ = (I - 2uu^T)^T(I - 2uu^T) = (I^T - 2(uu^T)^T)(I - 2uu^T)

= (I - 2(u^T)^Tu^T)(I - 2uu^T) = (I - 2uu^T)(I - 2uu^T)

= I \cdot I - 2Iuu^T - 2uu^TI + 4(uu^T)(uu^T) = I - 4uu^T +4u(u^Tu)u^T

= I - 4uu^T + 4uIu^T = I - 4uu^T + 4uu^T = I

Since Q^TQ = I the matrix Q = I - 2uu^T is orthogonal.

If u = (\frac{1}{2},\frac{1}{2}, -\frac{1}{2}, -\frac{1}{2}) then

uu^T = \begin{bmatrix} \frac{1}{2} \\ \frac{1}{2} \\ -\frac{1}{2} \\ -\frac{1}{2} \end{bmatrix} \begin{bmatrix} \frac{1}{2}&\frac{1}{2}&-\frac{1}{2}&-\frac{1}{2} \end{bmatrix} = \begin{bmatrix} \frac{1}{4}&\frac{1}{4}&-\frac{1}{4}&-\frac{1}{4} \\ \frac{1}{4}&\frac{1}{4}&-\frac{1}{4}&-\frac{1}{4} \\ -\frac{1}{4}&-\frac{1}{4}&\frac{1}{4}&\frac{1}{4} \\ -\frac{1}{4}&-\frac{1}{4}&\frac{1}{4}&\frac{1}{4} \end{bmatrix}

so that

Q = I - 2uu^T = \begin{bmatrix} 1&0&0&0 \\ 0&1&0&0 \\ 0&0&1&0 \\ 0&0&0&1 \end{bmatrix} - 2 \cdot \begin{bmatrix} \frac{1}{4}&\frac{1}{4}&-\frac{1}{4}&-\frac{1}{4} \\ \frac{1}{4}&\frac{1}{4}&-\frac{1}{4}&-\frac{1}{4} \\ -\frac{1}{4}&-\frac{1}{4}&\frac{1}{4}&\frac{1}{4} \\ -\frac{1}{4}&-\frac{1}{4}&\frac{1}{4}&\frac{1}{4} \end{bmatrix}

= \begin{bmatrix} 1&0&0&0 \\ 0&1&0&0 \\ 0&0&1&0 \\ 0&0&0&1 \end{bmatrix} - \begin{bmatrix} \frac{1}{2}&\frac{1}{2}&-\frac{1}{2}&-\frac{1}{2} \\ \frac{1}{2}&\frac{1}{2}&-\frac{1}{2}&-\frac{1}{2} \\ -\frac{1}{2}&-\frac{1}{2}&\frac{1}{2}&\frac{1}{2} \\ -\frac{1}{2}&-\frac{1}{2}&\frac{1}{2}&\frac{1}{2} \end{bmatrix} = \begin{bmatrix} \frac{1}{2}&-\frac{1}{2}&\frac{1}{2}&\frac{1}{2} \\ -\frac{1}{2}&\frac{1}{2}&\frac{1}{2}&\frac{1}{2} \\ \frac{1}{2}&\frac{1}{2}&\frac{1}{2}&-\frac{1}{2} \\ \frac{1}{2}&\frac{1}{2}&-\frac{1}{2}&\frac{1}{2} \end{bmatrix}

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.4

Exercise 3.4.4. Given two orthogonal matrices Q_1 and Q_2, show that their product Q_1Q_2 is also orthogonal. If Q_1 represents rotation through the angle \theta and Q_1 represents rotation through the angle \phi, what does Q_1Q_2 represent? What trigonometric identities for \sin (\theta + \phi) and \cos (\theta + \phi) can be found in multiplying Q_1 and Q_2?

Answer: For any orthogonal matrix Q we have Q^TQ = I. We then have

(Q_1Q_2)^T(Q_1Q2) = (Q_2^TQ_1^T)(Q_1Q2) = Q_2^T(Q_1^TQ_1)Q_2 = Q_2^TQ_2 = I

Since (Q_1Q_2)^T(Q_1Q2) = I the matrix Q_1Q_2 is orthogonal.

If the matrices Q_1 and Q_2 represent rotations through \theta and \phi respectively, then Q_1Q_2 represents rotation through \theta + \phi, and will contain elements including \sin (\theta + \phi) and \cos (\theta + \phi). These will be produced in the matrix multiplication through the following trigonometric identities:

\sin (\theta + \phi) = \sin \theta \cos \phi + \cos \theta \sin \phi

\cos (\theta + \phi) = \cos \theta \cos \phi - \sin \theta \sin \phi

where \sin \theta and \cos \theta come from Q_1  and \sin \phi and \cos \phi come from Q_2.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged , | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.3

Exercise 3.4.3. Given the orthonormal vectors a_1 = (\frac{2}{3}, \frac{2}{3}, -\frac{1}{3}) and a_2 = (-\frac{1}{3}, \frac{2}{3}, \frac{2}{3}) and the vector b = (0, 3, 0) from the previous exercise, project b onto a third orthonormal vector a_3 = (\frac{2}{3}, -\frac{1}{3}, \frac{2}{3}). What is the sum of the three projections? Why? Why is the matrix P = a_1a_1^T + a_2a_2^T + a_3a_3^T equal to the identity matrix I?

Answer: Since a_3 is orthonormal, the projection of b onto a_3 is given by

a_3^Tba_3 = \begin{bmatrix} \frac{2}{3}&-\frac{1}{3}&\frac{2}{3} \end{bmatrix} \begin{bmatrix} 0 \\ 3 \\ 0 \end{bmatrix} \begin{bmatrix} \frac{2}{3} \\ -\frac{1}{3} \\ \frac{2}{3} \end{bmatrix} = -1 \cdot \begin{bmatrix} \frac{2}{3} \\ -\frac{1}{3} \\ \frac{2}{3} \end{bmatrix} = \begin{bmatrix} -\frac{2}{3} \\ \frac{1}{3} \\ -\frac{2}{3} \end{bmatrix}

The sum of all three projections is then:

a_1^Tba_1 + a_2^Tba_2 + a_3^Tba_3 = \begin{bmatrix} \frac{4}{3} \\ \frac{4}{3} \\ -\frac{2}{3} \end{bmatrix} + \begin{bmatrix} -\frac{2}{3} \\ \frac{4}{3} \\ \frac{4}{3} \end{bmatrix} + \begin{bmatrix} -\frac{2}{3} \\ \frac{1}{3} \\ -\frac{2}{3} \end{bmatrix} = \begin{bmatrix} 0 \\ 3 \\ 0 \end{bmatrix} = b

This is because the vectors a_1, a_2, and a_3 together form an orthonormal basis for \mathbb{R}^3, and any vector v in \mathbb{R}^3 can be expressed as a linear combination of those basis vectors with coefficients given by the projection of v onto each basis vector.

We then have

b = a_1^Tba_1 + a_2^Tba_2 + a_3^Tba_3 = a_1a_1^Tb + a_2a_2^Tb + a_3a_3^Tb = (a_1a_1^T + a_2a_2^T + a_3a_3^T)b

(taking advantage of the fact that a_1^Tb, etc., are scalars) so that the 3 by 3 matrix

P = a_1a_1^T + a_2a_2^T + a_3a_3^T = I

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged , , | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.2

Exercise 3.4.2. Given two orthonormal vectors a_1 = (\frac{2}{3}, \frac{2}{3}, -\frac{1}{3}) and a_2 = (-\frac{1}{3}, \frac{2}{3}, \frac{2}{3}) and the vector b = (0, 3, 0), project b onto a_1 and a_2. Also find the projection p of b onto the plane formed by a_1 and a_2.

Answer: Since a_1 is orthonormal, the projection of b onto a_1 is given by

a_1^Tba_1 = \begin{bmatrix} \frac{2}{3}&\frac{2}{3}&-\frac{1}{3} \end{bmatrix} \begin{bmatrix} 0 \\ 3 \\ 0 \end{bmatrix} \begin{bmatrix} \frac{2}{3} \\ \frac{2}{3} \\ -\frac{1}{3} \end{bmatrix} = 2 \cdot \begin{bmatrix} \frac{2}{3} \\ \frac{2}{3} \\ -\frac{1}{3} \end{bmatrix} = \begin{bmatrix} \frac{4}{3} \\ \frac{4}{3} \\ -\frac{2}{3} \end{bmatrix}

Similarly the projection of b onto a_2 is given by

a_2^Tba_2 = \begin{bmatrix} -\frac{1}{3}&\frac{2}{3}&\frac{2}{3} \end{bmatrix} \begin{bmatrix} 0 \\ 3 \\ 0 \end{bmatrix} \begin{bmatrix} -\frac{1}{3} \\ \frac{2}{3} \\ \frac{2}{3} \end{bmatrix} = 2 \cdot \begin{bmatrix} -\frac{1}{3} \\ \frac{2}{3} \\ \frac{2}{3} \end{bmatrix} = \begin{bmatrix} -\frac{2}{3} \\ \frac{4}{3} \\ \frac{4}{3} \end{bmatrix}

The projection p of b onto the plane formed by a_1 and a_2 is simply the sum of the projections above:

p = a_1^Tba_1 + a_2^Tba_2 = \begin{bmatrix} \frac{4}{3} \\ \frac{4}{3} \\ -\frac{2}{3} \end{bmatrix} + \begin{bmatrix} -\frac{2}{3} \\ \frac{4}{3} \\ \frac{4}{3} \end{bmatrix} = \begin{bmatrix} \frac{2}{3} \\ \frac{8}{3} \\ \frac{2}{3} \end{bmatrix}

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged , | Leave a comment

Linear Algebra and Its Applications, Exercise 3.4.1

Exercise 3.4.1. a) Given the following four data points:

y = -4 \quad\textrm{at}\quad t = -2 \quad\textrm{and}\quad y = -3 \quad\textrm{at}\quad t = -1

y = -1 \quad\textrm{at}\quad t = 1 \quad\textrm{and}\quad y = 0 \quad\textrm{at}\quad t = 2

write down the four equations for fitting C + Dt to the data.

b) Find the line fit by least squares and calculate the error E^2.

c) Given the value of E^2 what is b in relation to the column space? What is the projection p of b on the column space?

Answer: a) This corresponds to a system of the form Ax = b as follows:

\begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&1 \\ 1&2 \end{bmatrix} \begin{bmatrix} C \\ D \end{bmatrix} = \begin{bmatrix} -4 \\ -3 \\ -1 \\ 0 \end{bmatrix}

The equivalent system of equations is:

C - 2D = -4

C - D = -3

C + D = -1

C + 2D = 0

b) To find the least squares solution we multiply both sides by A^T to create a system of the form A^TA\bar{x} = A^Tb. We have

A^TA = \begin{bmatrix} 1&1&1&1 \\ -2&-1&1&2 \end{bmatrix} \begin{bmatrix} 1&-2 \\ 1&-1 \\ 1&1 \\ 1&2 \end{bmatrix} = \begin{bmatrix} 4&0 \\ 0&10 \end{bmatrix}

and

A^Tb = \begin{bmatrix} 1&1&1&1 \\ -2&-1&1&2 \end{bmatrix} \begin{bmatrix} -4 \\ -3 \\ -1 \\ 0 \end{bmatrix} = \begin{bmatrix} -8 \\ 10 \end{bmatrix}

so that the new system is

\begin{bmatrix} 4&0 \\ 0&10 \end{bmatrix} \begin{bmatrix} \bar{C} \\ \bar{D} \end{bmatrix} = \begin{bmatrix} -8 \\ 10 \end{bmatrix}

From the second equation we have \bar{D} = \frac{10}{10} = 1 and from the first equation we have \bar{C} = \frac{-8}{4} = -2.

The resulting graph is a line -2 + t with slope of 1 and y-intercept of -2. For the values of t of -2, -1, 1, and 2 the values of -2 + t are -4, -3, -1, and 0 respectively. But these are exactly the same as the values of y in the given data points. Therefore we have E^2 = 0.

c) Since E^2 = 0 the vector b must be in the column space of A, and the projection p of b onto the column space is simply b itself.

NOTE: This continues a series of posts containing worked out exercises from the (out of print) book Linear Algebra and Its Applications, Third Edition by Gilbert Strang.

If you find these posts useful I encourage you to also check out the more current Linear Algebra and Its Applications, Fourth Edition, Dr Strang’s introductory textbook Introduction to Linear Algebra, Fifth Edition and the accompanying free online course, and Dr Strang’s other books.

Posted in linear algebra | Tagged , , | Leave a comment