Section 1.6 Projectors
Subsection 1.6.1 Oblique Projectors
Definition 1.6.1.
A square matrix P is a projector if P2=P.
Lemma 1.6.2. Projectors Fix Column Space.
Suppose P is a projector and xβC(P). Then Pxβx=0.
Proof.
Since \(\vect{x}\in\csp{P}\text{,}\) there is a vector \(\vect{w}\) such that \(P\vect{w}=\vect{x}\text{.}\) Then
Lemma 1.6.3. Projector Directions are Null Space.
Suppose P is a projector of size n and xβCn is any vector. Then PxβxβN(P). Furthermore, N(P)={Pxβx|xβCn}.
Proof.
First,
To establish the second half of the claimed set equality, suppose \(\vect{z}\in\nsp{P}\text{,}\) then
which establishes that \(\vect{z}\in\setparts{P\vect{x}-\vect{x}}{\vect{x}\in\complex{n}}\text{.}\)
Example 1.6.4. Projector in Three Dimensions.
Verify the following facts about the matrix \(P\) to understand that it is a projector and to understand its geometry.
\(\displaystyle P^2=P\)
\(\displaystyle \csp{P}=\spn{\set{\colvector{1\\0\\\frac{-2}{5}},\,\colvector{0\\1\\\frac{-3}{5}}}}\)
\(\displaystyle \nsp{P}=\spn{\set{\colvector{1\\2\\1}}}\)
So \(P\) sends every vector onto a two-dimensional subspace, with an equation we might write as \(2x+3y+5z=0\) in Cartesian coordinates, or which we might describe as the plane through the origin with normal vector \(\vect{n}=2\vec{i}+3\vec{j}+5\vec{k}\text{.}\) Vectors, or points, are always moved in the direction of the vector \(\vect{d}=\vec{i}+2\vec{j}+1\vec{k}\)βthis is the direction the light is shining. Exercise Checkpoint 1.6.5 asks you to experiment further.
Checkpoint 1.6.5.
Continue experimenting with Example Example 1.6.4 by constructing a vector not in the column space of \(P\text{.}\) Compute its image under \(P\) and verify that it is a linear combination of the basis vectors given in the example. Compute the direction your vector moved and verify that it is a scalar multiple of the basis vector for the null space given in the example. Finally, construct a new vector in the column space and verify that it is unmoved by \(P\text{.}\)
Definition 1.6.6.
Given a projector P, the complementary projector to P is IβP.
Lemma 1.6.7. Complementary Projector is a Projector.
If P is a projector then IβP is also a projector.
Proof.
Lemma 1.6.8. Complementary Projector's Column Space.
Suppose P is a projector. Then C(IβP)=N(P) and therefore N(IβP)=C(P).
Proof.
First, suppose \(\vect{x}\in\nsp{P}\text{.}\) Then
demonstrating that \(\vect{x}\) is a linear combination of the columns of \(I-P\text{.}\) So \(\nsp{P}\subseteq\csp{I-P}\text{.}\)
Now, suppose \(\vect{x}\in\csp{I-P}\text{.}\) Then there is a vector \(\vect{w}\) such that \(\vect{x}=\left(I-P\right)\vect{w}\text{.}\) Then
So \(\csp{I-P}\subseteq\nsp{P}\text{.}\)
To establish the second conclusion, replace the projector \(P\) in the first conclusion by the projector \(I-P\text{.}\)
Theorem 1.6.9. Projector Vector Space Decomposition.
Suppose P is a projector of size n. Then Cn=C(P)βN(P).
Proof.
First, we show that \(\csp{P}\cap\nsp{P}=\set{\zerovector}\text{.}\) Suppose \(\vect{x}\in\csp{P}\cap\nsp{P}\text{.}\) Since \(\vect{x}\in\csp{P}\text{,}\) Lemma Lemma 1.6.8 implies that \(\vect{x}\in\nsp{I-P}\text{.}\) So
Using Lemma Lemma 1.6.8 again, \(\nsp{P}=\csp{I-P}\text{.}\) We show that an arbitrary vector \(\vect{w}\in\complex{n}\) can be written as a sum of two vectors from the two column spaces,
So \(\complex{n}\text{,}\) \(\csp{P}\) and \(\nsp{P}\) meet the hypotheses of Theorem Theorem 1.2.5, allowing us to establish the direct sum.
Subsection 1.6.2 Orthogonal Projectors
The projectors of the previous section would be termed oblique projectors since no assumption was made about the direction that a vector was moved when projected. We remedy that situation now by defining an orthogonal projector to be a projector where the complementary subspace is orthogonal to the space the projector projects onto.Definition 1.6.10.
A projector P is orthogonal if N(P)=(C(P))β₯.
Theorem 1.6.11. Orthogonal Projectors are Hermitian.
Suppose P is a projector. Then P is an orthogonal projector if and only if P is Hermitian.
Proof.
Theorem HMIP says that a Hermitian matrix \(A\) is characterized by the property that \(\innerproduct{A\vect{x}}{\vect{y}}=\innerproduct{\vect{x}}{A\vect{y}}\) for every choice of the vectors \(\vect{x}, \vect{y}\text{.}\) We will use this result in both halves of the proof.
Suppose that \(\vect{x}\in\nsp{P}\text{.}\) Then for any \(\vect{y}\in\csp{P}\text{,}\) there is a vector \(\vect{w}\) that allows us to write
So \(\nsp{P}\subseteq\per{\csp{P}}.\)
Now suppose that \(\vect{x}\in\per{\csp{P}}\text{.}\) Consider,
By Theorem PIP, we conclude that \(P\vect{x}=\zerovector\) and \(\vect{x}\in\nsp{P}\text{.}\) So \(\per{\csp{P}}\subseteq\nsp{P}\) and we have establish the set equality of Definition Definition 1.6.10.
Let \(\vect{u},\vect{v}\in\complex{n}\) be any two vectors. Decompose each into two pieces, the first from the column space, the second from the null space, according to Theorem Theorem 1.6.9. So
with \(\vect{u}_1,\vect{v}_1\in\csp{P}\) and \(\vect{u}_2,\vect{v}_2\in\nsp{P}\text{.}\) Then
Since \(\innerproduct{P\vect{u}}{\vect{v}}=\innerproduct{\vect{u}}{P\vect{v}}\) for all choices of \(\vect{u},\vect{v}\in\complex{n}\text{,}\) Theorem HMIP, establishes that \(P\) is Hermitian.
[provisional cross-reference: adjoint-A is nonsingular result]
), so we can employ its inverse to find
Theorem 1.6.12. Orthogonal Projector Construction.
Suppose U is a subspace and A is a matrix whose columns form a basis of U. Then P=A(AβA)β1Aβ is an orthogonal projector onto U.
Proof.
Because \(A\) is the leftmost term in the product for \(P\text{,}\) \(\csp{P}\subseteq\csp{A}\text{.}\) Because \(\inverse{\left(\adjoint{A}A\right)}\adjoint{A}\) has full (column) rank, \(\csp{A}\subseteq\csp{P}\text{.}\) So the image of the projector is exactly \(U\text{.}\)
Now we verify that \(P\) is a projector.
And lastly, orthogonality against a basis of \(U\text{.}\)
Checkpoint 1.6.13.
Illustrate Theorem Theorem 1.6.11 by proving directly that the orthogonal projector described in Theorem Theorem 1.6.12 is Hermitian.
Checkpoint 1.6.14.
Construct the orthogonal projector onto the line spanned by
Illustrate its use by projecting some vector not on the line, and verifying that the difference between the vector and its projection is orthogonal to the line.
Checkpoint 1.6.15.
Construct the orthogonal projector onto the subspace
Illustrate its use by projecting some vector not in the subspace, and verifying that the difference between the vector and its projection is orthogonal to the line.
Checkpoint 1.6.16.
Redo Exercise Checkpoint 1.6.15 but first convert the basis for \(U\) to an orthonormal basis via the Gram-Schmidt process Theorem GSP and then use the simpler construction applicable to the case of an orthonormal basis.