a Notes for Linear Algebra Done Right——魏公子 Chapter1. Vector Space 1.A Complex Number and Cn${\displaystyle {\mathbb{C}}^n}$ 1.B Definition of Vector Space 1.C Subspaces Definition Sums of Subspaces Chapter2. Finite-Dimensional Vector Space 2.A Span and Linear Independence Linear Combinations and Span Linear Independence 2.B Bases 2.C Dimension Chapter3. Linear Maps 3.A The Vector Space of Linear Maps Definition and Examples of Linear Maps Algebraic Operations on L(V,W)${\displaystyle \mathcal{L}(V,W)}$ 3.B Null Spaces and Ranges Null Spaces and Injectivity Range and Surjectivity Fundamental Theorem of Linear Maps 3.C Matrices Representing a Linear Map by a Matrix 3.D Invertibility and Isomorphic Vector Spaces Invertible Linear Maps Isomorphic Vector Spaces Linear Maps thought of as Matrix Multiplication Operators 3.E Products and Quotients of Vector Spaces Products of Vector Spaces Products and Direct Sums Quotients of Vector Space 3.F Duality The Dual Space and the Dual Map The null Space and Range of the Dual of a Linear Map The Matrix of the Dual map of a Linear Map The Rank of a Matrix Chapter4. Polynomials 4.A polynomials The Division Algorithm for Polynomials Chapter5. Eigenvalues, Eigenvectors, and Invariant Subspaces 5.A Invariant Subspaces Eigenvalues and Eigenvectors Restriction and Quotient Operators 5.B Eigenvectors and Upper-Triangular Matrices Polynomials Applied to Operators Existence of Eigenvalues Upper-Triangular Matrices 5.C Eigenspaces and Diagonal Matrices Chapter6. Inner Product Spaces 6.A Inner Products and Norms Inner Products Norms 6.B Orthonormal Bases Linear Functional on Inner Product Spaces 6.C Orthogonal Complements and Minimization Problems Orthogonal Complements Minimization Problems Chapter7. Operators on Inner Product Spaces 7.A Self-Adjoint and Normal Operators Adjoints Normal Operators 7.B The spectral Theorem The Complex Spectral Theorem The Real Spectral Theorem 7.C Positive Operators and Isometries Positive Operators Isometries (unitary operator) 7.D Polar Decomposition and Singular Value Decomposition Polar decomposition Singular Value Decomposition Chapter8. Operators on Complex Vector Spaces 8.A Generalized Eigenvectors and Nilpotent Operators Null Spaces of Powers of an Operator Generalized Eigenvectors Nilpotent Operators 8.B Decomposition of an Operator Description of Operators on Complex Vector Spaces Multiplicity of an Eigenvalue Block Diagonal Matrices Square Roots 8.C Characteristic and Minimal Polynomial The Cayley-Hamilton Theorem The Minimal Polynomial 8.D Jordan Form Chapter9. Operators on Real Vector Spaces 9.A Complexification of a Vector Space Complexification of an Operator The Minimal Polynomial of the Complexification Eigenvalues of the Complexification Chapter10. Trace and Determinant 10.A Trace Change of Basis Trace: A Connection Between Operators and Matrices 10.B Determinant Determinant of an operator Determinant of a matrix Chapter1. Vector Space1.A Complex Number and Cn${\displaystyle {\mathbb{C}}^n}$Definition1.A.1subtractionlet -𝛼${\displaystyle -𝛼}$ denotes the complex number which satisfies:𝛼 + (-𝛼) = 0$$𝛼+(-𝛼)=0$$then subtraction is defined by :𝛽-𝛼:=𝛽+(-𝛼)$$𝛽-𝛼:=𝛽+(-𝛼)$$□• Division is defined similarly Definition1.A.2Notation F${\displaystyle \mathbb{F}}$: F${\displaystyle \mathbb{F}}$ stands for either R ${\displaystyle \mathbb{R}}$or C${\displaystyle \mathbb{C}}$

Definition1list and length

A list of length n${\displaystyle n}$ is (must be finite, because it has a length)(x1, x2,⋯, xn)$$(x_1,x_2,\cdots ,x_n)$$Many mathematicians call a list of length n an n-tuple.

Definition1.A.1 Fn${\displaystyle {\mathbb{F}}^n}$Definition1.A.20Let 0 denote the list of length n whose coordinates are all 0: (0,⋯,0)${\displaystyle (0,\cdots ,0)}$1.B Definition of Vector Space

Definition1Notation FS${\displaystyle {\mathbb{F}}^S}$

A vector space is a set V${\displaystyle V}$, along with an addition on V ${\displaystyle V}$and a scalar multiplication on V${\displaystyle V}$ such that the following properties hold: • commutativityu+v=v+u$$u+v=v+u$$• associativity(u+v)+w=u+(v+w)$$(u+v)+w=u+(v+w)$$• additive identityv+0=v $$v+0=v$$• additive inversev+w=0$$v+w=0$$• multiplicative identity1⋅v=v$$1\cdot v=v$$• distributive propertiesa(u+v)=au+av$$a(u+v)=au+av$$ infinite vector space:• F∞${\displaystyle {\mathbb{F}}^{\infty }}$ is defined to be the set of all sequences of elements of F${\displaystyle \mathbb{F}}$:F∞={(x1,x2,⋯):xj∈F for j=1,2,⋯}$${\mathbb{F}}^{\infty }=\{(x_1,x_2,\cdots ):x_j\in \mathbb{F}forj=1,2,\cdots \}$$

Definition2Notation FS${\displaystyle {\mathbb{F}}^S}$

• If S${\displaystyle S}$ is a set, then FS${\displaystyle {\mathbb{F}}^S}$ denotes the set of functions from S${\displaystyle S}$ to F${\displaystyle \mathbb{F}}$• For f, g∈FS${\displaystyle f,g\in {\mathbb{F}}^S}$, the sum f+g∈FS${\displaystyle f+g\in {\mathbb{F}}^S}$ is the function defined by (f+g)(x)=f(x)+g(x)$$(f+g)(x)=f(x)+g(x)$$• For 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$ and f∈FS${\displaystyle f\in {\mathbb{F}}^S}$, the product 𝜆f∈FS${\displaystyle 𝜆f\in {\mathbb{F}}^S}$ is the function defined by(𝜆f)(x)=𝜆f(x)$$(𝜆f)(x)=𝜆f(x)$$• for all x∈S.${\displaystyle x\in S.}$

For example, R[0,1]${\displaystyle {\mathbb{R}}^{[0,1]}}$ is the set of real-values functions on the interval [0,1].${\displaystyle [0,1].}$But, wait!

Example2.1FS${\displaystyle {\mathbb{F}}^S}$ is a vector space

We just need to define:• The additive identity of FS${\displaystyle {\mathbb{F}}^S}$ is the function 0:S→F${\displaystyle 0:S\to \mathbb{F}}$ defined by0(x)=0$$0(x)=0$$• The additive inverse of f${\displaystyle f}$ is the function -f: S→F${\displaystyle -f:S\to \mathbb{F}}$ defined by(-f)(x):=-f(x)$$(-f)(x):=-f(x)$$• for all x∈S.${\displaystyle x\in S.}$

1.C SubspacesDefinition

Definition3subspaces

A subset U${\displaystyle U}$ of V${\displaystyle V}$ is called a subspace if U${\displaystyle U}$ is also a vector spaces, using the same addition and scalar multiplication.

Sums of Subspaces

Definition4sum of subsets

Suppose U1,⋯,Um${\displaystyle U_1,\cdots ,U_m}$ are subsets of V.${\displaystyle V.}$ The sum of U1,⋯Um${\displaystyle U_1,\cdots U_m}$ , denoted U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$, is the set of all possible sums of elements of U1,⋯,Um${\displaystyle U_1,\cdots ,U_m}$. More precisely,U1+⋯+Um:={u1+⋯+um:u1∈U1,⋯,um∈Um}$$U_1+\cdots +U_m:=\{u_1+\cdots +u_m:u_1\in U_1,\cdots ,u_m\in U_m\}$$

Example4.1

Suppose U${\displaystyle U}$ is the set of all elements of F3${\displaystyle {\mathbb{F}}^3}$ whose second and third coordinates equal 0, and W${\displaystyle W}$ is the set of all elements of F3${\displaystyle {\mathbb{F}}^3}$ whose first and third coordinates equal 0:U={(x, 0, 0)∈F3:x∈F} and W={(0, y, 0)∈F3:y∈F}$$U=\{(x,0,0)\in {\mathbb{F}}^3:x\in \mathbb{F}\}andW=\{(0,y,0)\in {\mathbb{F}}^3:y\in \mathbb{F}\}$$ThenU+W={(x,y,0)∈F3: x,y∈F}$$U+W=\{(x,y,0)\in {\mathbb{F}}^3:x,y\in \mathbb{F}\}$$

Theorem5Sum of subspaces is the smallest containing subspace

Suppose U1,⋯Um${\displaystyle U_1,\cdots U_m}$ are subspaces of V${\displaystyle V}$. Then U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ is the smallest subspace of V${\displaystyle V}$ containing U1,⋯,Um${\displaystyle U_1,\cdots ,U_m}$.Proof : • First it's easy to prove U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ is a subspace of V. • Then, every subspace in U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ containing U1,⋯,Um${\displaystyle U_1,\cdots ,U_m}$ must contain U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ because subspaces must contain all finite sums of their elements (closed under addition), so if it contains U1${\displaystyle U_1}$ and U2${\displaystyle U_2}$, then the element of U1+U2${\displaystyle U_1+U_2}$ must be contained to.

Now, what about the situation each vector in U1+⋯Um${\displaystyle U_1+\cdots U_m}$ can be represented in the from above in only one way?

Definition6Direct Sum

Suppose U1,⋯Um${\displaystyle U_1,\cdots U_m}$ are subspaces of V${\displaystyle V}$. • Then U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ is called a direct sum if each element of U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ can be written in only one way as a sum u1+⋯um${\displaystyle u_1+\cdots u_m}$, where each uj${\displaystyle u_j}$ is in Uj${\displaystyle U_j}$• If U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ is a direct sum, then U1⊕⋯⊕Un${\displaystyle U_1\oplus \cdots \oplus U_n}$ denotes U1+⋯Um${\displaystyle U_1+\cdots U_m}$, with the ⊕${\displaystyle \oplus }$ notation serving as an indication that this is a direct sum.

Example6.1direct sum

Suppose U${\displaystyle U}$ is the subspace of F3${\displaystyle {\mathbb{F}}^3}$ of those vectors whose last coordinates equal 0, and W${\displaystyle W}$ is the subspace of F3${\displaystyle {\mathbb{F}}^3}$ of those vectors whose first two coordinates equal 0:U={(x,y,0)∈F3:x,y∈F} and W={(0,0,z)∈F3: z∈F}.$$U=\{(x,y,0)\in {\mathbb{F}}^3:x,y\in \mathbb{F}\}andW=\{(0,0,z)\in {\mathbb{F}}^3:z\in \mathbb{F}\}.$$The F3=U⊕W${\displaystyle {\mathbb{F}}^3=U\oplus W}$.

The definition of direct sum requires that every vector in the sum have a unique representation as an appropriate sum. The next result shows that when deciding whether a sum is direct, you only need consider whether 0 can be uniquely written as an appropriate sum.

Theorem7The condition of direct sum

Suppose U1,⋯Um${\displaystyle U_1,\cdots U_m}$ are subspaces of V${\displaystyle V}$. Then U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ is a direct sum if and only if the only way to write 0${\displaystyle 0}$ as a sum u1+⋯+um${\displaystyle u_1+\cdots +u_m}$, where each uj${\displaystyle u_j}$ is in Uj${\displaystyle U_j}$, is by taking each uj${\displaystyle u_j}$ equal to 0${\displaystyle 0}$.Proof • First suppose U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ is a direct sum. Then the definition requires 0 can only be represented by one way. Then because each Uj${\displaystyle U_j}$ is a subspace of U${\displaystyle U}$, so it means 0∈Uj${\displaystyle 0\in U_j}$, thus 0+⋯+0=0${\displaystyle 0+\cdots +0=0}$ is possible. So the only way to write 0 as a sum is it.• If we can write 0 in the only way, then we can't write a vector in two ways:v=u1+⋯um=𝜈1+⋯𝜈m$$v=u_1+\cdots u_m={𝜈}_1+\cdots {𝜈}_m$$• Subtracting these two equations, we have0=(u1-𝜈1)+⋯(um-𝜈m)$$0=(u_1-{𝜈}_1)+\cdots (u_m-{𝜈}_m)$$• Because 0=0+⋯+0${\displaystyle 0=0+\cdots +0}$, sou1=𝜈1, u2=𝜈2,⋯□$$u_1={𝜈}_1,u_2={𝜈}_2,\cdots$$

The next result gives a simple condition for testing which pairs of subspaces give a direct sum.

Theorem8Direct sum of two subspaces

Suppose U${\displaystyle U}$ and W${\displaystyle W}$ are subspaces of V${\displaystyle V}$. Then U+W${\displaystyle U+W}$ is a direct sum if and only if U∩W={0}${\displaystyle U\cap W=\{0\}}$Proof If it's a direct sum, then, there can't be other representation of 0 except 0+0${\displaystyle 0+0}$. If U∩W={0,s}${\displaystyle U\cap W=\{0,s\}}$, then because of the addition conservation, there must be a -s${\displaystyle -s}$ in W${\displaystyle W}$ as well. So there can be another way to represent:0=s+(-s)$$0=s+(-s)$$ Besides, it's easy to prove that if U∩W={0}${\displaystyle U\cap W=\{0\}}$, there is only one way to represent 0. Thus Theorem "Condition of direct sum" is satisfied.□

More information

The results above only deals with the case of two subspaces. When asking about a possible direct sum with more than two subspaces, it is not enough to test whether U1∩U2=U1∩U3=U2∩U3={0}${\displaystyle U_1\cap U_2=U_1\cap U_3=U_2\cap U_3=\{0\}}$

Chapter2. Finite-Dimensional Vector Space2.A Span and Linear IndependenceTo avoid confusion, we will usually write lists of vectors without surrounding parentheses. For example: (4,1,6), (9,5,7) is a list of length 2 of vectors in R3${\displaystyle {\mathbb{R}}^3}$Linear Combinations and SpanAdding up scalar multiples of vectors in a list gives what is called a linear combination of the list:

Definition9Linear Combination

A linear combination of a list v1,⋯vm${\displaystyle v_1,\cdots v_m}$ of vectors in V${\displaystyle V}$ is a vector of the forma1v1+⋯+amvm$$a_1{v}_1+\cdots +a_m{v}_m$$where a1,⋯,am∈F${\displaystyle a_1,\cdots ,a_m\in \mathbb{F}}$.

Definition10Span

(Some mathematicians also call it linear span)

The set of all linear combination of a list of vectors v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ in V${\displaystyle V}$ is called the span of v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$, denoted span(v1,⋯,vm)${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_m)}$. In other words(1)span(v1,⋯,vm)={a1v1+⋯+amvm:a1,⋯,am∈F}$$\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_m)=\{a_1{v}_1+\cdots +a_m{v}_m:a_1,\cdots ,a_m\in \mathbb{F}\}$$The span of the empty list ( ) is defined to be {0}. Besides, we introduce a verb spans. We say that v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ spans V${\displaystyle V}$.

Theorem11Span is the smallest containing subspace

Proof First we show that span(v1,⋯vm)${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots v_m)}$ is a subspace of V${\displaystyle V}$. • The additive identity is in it: set all ai=0${\displaystyle a_i=0}$ in equ (1). • It's closed under addition• It's closed under scalar multiplicationThen we show that span${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}}$ is the smallest subspace• Each vj${\displaystyle v_j}$ is a linear combination of v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$, thus is in the span.• Conversely, if we need to contain all vectors in the list, we must allow the addition and scalar multiplication. Thus we must include the span if we want to include all the vectors in the list□

Now we can make one of the key definition in linear algebra!!!

Definition12finite-dimensional vector space

A vector space is called finite-dimensional if some list of vectors in it spans the space.

Recall that by definition every list has finite length.

Definition13polynomial, P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$

• A function p:F→F${\displaystyle p:\mathbb{F}\to \mathbb{F}}$ is called a polynomial with coefficients in F${\displaystyle \mathbb{F}}$ if there exist a0,⋯,am∈F${\displaystyle a_0,\cdots ,a_m\in \mathbb{F}}$ such thatp(z)=a0+a1z+a2z2+⋯+amzm$$p(z)=a_0+a_{1}z+a_2{z}^2+\cdots +a_m{z}^m$$• for all z∈F${\displaystyle z\in \mathbb{F}}$.• P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$ is the set of all polynomial with coefficients in F${\displaystyle \mathbb{F}}$.

With the usual operations of addition and scalar multiplication, P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$ is a vector space over F${\displaystyle \mathbb{F}}$. In other words, P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$ is a subspace of FF${\displaystyle {\mathbb{F}}^{\mathbb{F}}}$, the vector space of functions from F${\displaystyle \mathbb{F}}$ to F${\displaystyle \mathbb{F}}$. If a polynomial is represented by two sets of coefficients, then subtracting one representation of the polynomial from the other produces a polynomial that is identically zero as a function on F${\displaystyle \mathbb{F}}$ and hence has all zero coefficients. Conclusion: the coefficients of a polynomial are uniquely determined by the polynomial.

Definition14degree of a polynomial: deg p${\displaystyle \mathrm{d}\mathrm{e}\mathrm{g}p}$

A polynomial is said to have a degree m${\displaystyle m}$ if there exist am≠0${\displaystyle a_m\ne 0}$ such thatp(z)=a0+a1z+⋯amzm$$p(z)=a_0+a_{1}z+\cdots a_m{z}^m$$for all z∈F${\displaystyle z\in \mathbb{F}}$. If p${\displaystyle p}$ has degree m${\displaystyle m}$, we write deg p=m${\displaystyle \mathrm{d}\mathrm{e}\mathrm{g}p=m}$. ★ The polynomial that is identically 0 is called to have degree -∞${\displaystyle -\infty }$

In next definition, we use the convention that -∞<m${\displaystyle -\infty <m}$, which means that the polynomial 0${\displaystyle 0}$ is in any set of polynomials Pm(F)${\displaystyle {\mathcal{P}}_m(\mathbb{F})}$.

Definition15Pm(F)${\displaystyle {\mathcal{P}}_m(\mathbb{F})}$

For m${\displaystyle m}$ a non-negative integer, Pm(F)${\displaystyle {\mathcal{P}}_m(\mathbb{F})}$ denotes the set of all polynomials with coefficients in F${\displaystyle \mathbb{F}}$ and degree at most m${\displaystyle m}$.

Example15.1Pm(F)${\displaystyle {\mathcal{P}}_m(\mathbb{F})}$ is a finite-dimensional vector space.

Pm(F)${\displaystyle {\mathcal{P}}_m(\mathbb{F})}$ is a finite-dimensional vector space for each non-negative integer m${\displaystyle m}$. Note that Pm(F)=span(1,z,⋯,zm)${\displaystyle {\mathcal{P}}_m(\mathbb{F})=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(1,z,\cdots ,z^m)}$; here we are slightly abusing notation by letting zk${\displaystyle z^k}$ denote a function, so zm∈FF${\displaystyle z^m\in {\mathbb{F}}^{\mathbb{F}}}$, and with all the number smaller than m${\displaystyle m}$, they spans a bigger subspace Pm(F)${\displaystyle {\mathcal{P}}_m(\mathbb{F})}$ in the vector space FF${\displaystyle {\mathbb{F}}^{\mathbb{F}}}$.

However, P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$ is infinite-dimensional

Example15.2P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$ is an infinite-dimensional vector space.

Consider any list of elements in P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$, let m${\displaystyle m}$ denotes the highest degree in the list. Then the span of the list cannot represent zm+1${\displaystyle z^{m+1}}$

Linear IndependenceSuppose v1,⋯,vm∈V${\displaystyle v_1,\cdots ,v_m\in V}$ and v∈span(v1,⋯,vm)${\displaystyle v\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_m)}$. If 0∈span(vj)${\displaystyle 0\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_j)}$ can only be written as the combination of vj${\displaystyle v_j}$ in one trivial way——0v1+0v2+⋯${\displaystyle 0v_1+0v_2+\cdots }$, then this situation is so important that we give it a special name -- linear independence

Definition16Linearly Independence

A list v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ of vectors in V ${\displaystyle V}$is called linearly independent if the only choice of a1,...,am∈F${\displaystyle a_1,...,a_m\in \mathbb{F}}$ that makes a1v1+⋯+amvm${\displaystyle a_1{v}_1+\cdots +a_m{v}_m}$ equal 0 is a1=⋯=am=0${\displaystyle a_1=\cdots =a_m=0}$. ★ The empty list ( ) is also declared to be linearly independent.

The reasoning above shows that v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ is linearly independent if and only if each vector in span(v1,⋯,vm)${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_m)}$ has only one representation as a linear combination of vj${\displaystyle v_j}$ Now we define Linear Dependent

Definition17Linearly dependent

A list v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ of vectors in V ${\displaystyle V}$is called linearly dependent if it is not linearly independent. In other words, a list of vector is linearly dependent if there exist a1,⋯,am∈F${\displaystyle a_1,\cdots ,a_m\in \mathbb{F}}$, not all 0, such that a1v1+⋯amvm=0${\displaystyle a_1{v}_1+\cdots a_m{v}_m=0}$

Lemma17.1Linear Dependence Lemma

Suppose v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ is a linearly dependent list in V${\displaystyle V}$ . Then there exists j∈{1,2,⋯,m}${\displaystyle j\in \{1,2,\cdots ,m\}}$ such that the following hold:★ vj∈span(v1,⋯,vj-1)${\displaystyle v_j\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_{j-1})}$★ if the jth${\displaystyle j^{\mathrm{t}\mathrm{h}}}$ term is removed from v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$, the span of the remaining list equals previous span.The proof is easy. Just change the form of the 0=a1v1+⋯+amvm${\displaystyle 0=a_1{v}_1+\cdots +a_m{v}_m}$, and then use other vector in the list to present any of the vj${\displaystyle v_j}$

The second lemma above is very useful. It means most time, we can simply add any vector into the list and remove another:

Theorem18Length of linearly independent list ⩽${\displaystyle ⩽}$ length of spanning list

Proof Let the vector in the linearly independent list denoted by uj${\displaystyle u_j}$, those in the spanning list denoted by wj${\displaystyle w_j}$, up to wn${\displaystyle w_n}$. And we do the following steps:(1) Each time, we add a new uj${\displaystyle u_j}$ into the spanning list, thus we change it into a linear dependent list(2) Then, we remove any wj${\displaystyle w_j}$ of the list. Because uj${\displaystyle u_j}$ is independent.Then we create a spanning list of length n${\displaystyle n}$, any other uj${\displaystyle u_j}$ cannot be linearly independent with this spanning list.□

We can use these theorem to prove some "trivial" true:

Theorem19Finite-dimensional subspaces

Every subspaces of a finite-dimensional vector space is finite-dimensionalProof Let v0=0${\displaystyle v_0=0}$. If U=span(v1,⋯,vj-1)${\displaystyle U=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_{j-1})}$, then U${\displaystyle U}$ is finite-dimensional. Otherwise, choose a vector vj∈U${\displaystyle v_j\in U}$ such thatvj∉span(v1,⋯,vj-1)$$v_j\notin \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_{j-1})$$After each step, as long as the process continues, we have constructed a list of vectors such that no vector in this list is in the span of the previous vectors. Thus after each step, the list is linearly independent. By linear independent Lemma, This linearly independent list cannot be longer than any spanning list of V${\displaystyle V}$. □

2.B BasesIn the last section, we discussed linearly independent lists and spanning lists. Now we bring there concepts together.

Definition20basis

A basis of V${\displaystyle V}$ is a list of vectors in V${\displaystyle V}$ that is linearly independent and spans V${\displaystyle V}$.

Theorem21Criterion for basis

A list v1,⋯vn${\displaystyle v_1,\cdots v_n}$ of vectors in V${\displaystyle V}$ is a basis of V ${\displaystyle V}$ if and only if every v∈V${\displaystyle v\in V}$ can be written uniquely in the formv=a1v1+⋯+anvn$$v=a_1{v}_1+\cdots +a_n{v}_n$$where a1,⋯,an∈F${\displaystyle a_1,\cdots ,a_n\in \mathbb{F}}$.

Theorem22spanning list contains a basis

Proof ★ If spanning list is linearly independent, it's a basis★ Otherwise, remove any vector of it until it's linearly independent. □${\displaystyle }$

Theorem23Basis of finite-dimensional vector space

Any finite-dimensional vector space has a basis, because it has some list spanning it.

Our next theorem is in some sense a dual of Theorem "spanning list contains a basis"

Theorem24Linearly independent list extends to a basis

Every linearly independent list of vectors in a finite-dimensional vector space can be extended to a basis of the vector space.Proof Keep adding additional linearly independent vector into the list, until its length equal to the basis of the space. ——not formal We can simply add the basis to the independent list, keep removing vectors of it until it becomes a basis. ( Because the vector in previous list v1⋯vn${\displaystyle v_1\cdots v_n}$ is linearly independent, so we don't need to remove any vj${\displaystyle v_j}$)

As an application of the results above, we now show that every subspace of a finite-dimensional vector space can be paired with another subspace to form a direct sum of the whole space

Theorem25Every subspace of V${\displaystyle V}$ is part of sum equal to V${\displaystyle V}$

A list v1,⋯vn${\displaystyle v_1,\cdots v_n}$ of vectors in V${\displaystyle V}$ is a basis of V ${\displaystyle V}$ if and only if every v∈V${\displaystyle v\in V}$ can be written uniquely in the formv=a1v1+⋯+anvn$$v=a_1{v}_1+\cdots +a_n{v}_n$$where a1,⋯,an∈F${\displaystyle a_1,\cdots ,a_n\in \mathbb{F}}$.

2.C DimensionAlthough we have been discussing finite-dimensional vector space, what's the dimension of such an object? A reasonable definition should force the dimension of Fn${\displaystyle {\mathbb{F}}^n}$ to equal n${\displaystyle n}$. Notice that the standard basis(1,0,⋯,0),⋯$$(1,0,\cdots ,0),\cdots$$of Fn${\displaystyle {\mathbb{F}}^n}$ has length n${\displaystyle n}$. Thus we try to define the dimension as the length of a basis. However, before doing this, can we promise that every basis of a vector space has equal length?

Theorem26Basis length does not depend on basis

ProofSuppose V${\displaystyle V}$ is finite-dimensional. Let B1${\displaystyle B_1}$ and B2${\displaystyle B_2}$ be two bases. Then B1${\displaystyle B_1}$ is a linearly independent list in the view of B2${\displaystyle B_2}$. The length of B2${\displaystyle B_2}$ thus must bigger than length of B1${\displaystyle B_1}$. And vice versa.

Now we can formally define the dimension of such spaces:

Definition27dimension, dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V}$

The dimension of a finite-dimensional vector space is the length of any basis of the vector space.

Example27.1dim Pm(F)=m+1${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}{\mathcal{P}}_m(\mathbb{F})=m+1}$

A polynomial has dimension: dim Pm(F)=m+1${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}{\mathcal{P}}_m(\mathbb{F})=m+1}$,because the basis is1,z,⋯zm$$1,z,\cdots z^m$$

To check that a list of vectors in V${\displaystyle V}$ is a basis of V${\displaystyle V}$ , we must, according to the definition, show that the list in question satisfies two properties. However, sometimes it's easier than this:

Theorem28Linearly independent list of right length is a basis

ProofSuppose dim V=n${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V=n}$ and v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is linearly independent in V${\displaystyle V}$. The list v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ can be extended to the basis, but basis has length equal n${\displaystyle n}$, thus the list itself is already a basis.

Example28.1Show that list (5,7), (4,3)${\displaystyle (5,7),(4,3)}$ is a basis of F2${\displaystyle {\mathbb{F}}^2}$

It's easy to show. Just remember that two vector with "," is a list.

Similarly, we have

Theorem29Spanning list of right length is a basis

ProofSpanning list contains a basis. But all bases have the same length. Thus the list itself is a basis.

The next result gives a formula for the dimension of the sum of two subspaces of a finite-dimensional vector space. The formula is analogous to a familiar counting formula: the number of elements in the union of two set equals the number of elements in the first set, plus the number of elements in the second set , minus the number of element in the intersection of the two set.

Theorem30Dimension of a sum

If U1${\displaystyle U_1}$ and U2${\displaystyle U_2}$ are subspaces of a finite-dimensional vector space, thendim(U1+U2)=dim U1+dim U2-dim(U1∩U2).$$\mathrm{d}\mathrm{i}\mathrm{m}(U_1+U_2)=\mathrm{d}\mathrm{i}\mathrm{m}{U}_1+\mathrm{d}\mathrm{i}\mathrm{m}{U}_2-\mathrm{d}\mathrm{i}\mathrm{m}(U_1\cap U_2).$$ProofLet u1,⋯,um${\displaystyle u_1,\cdots ,u_m}$ be the basis of U1∩U2${\displaystyle U_1\cap U_2}$, because u1,⋯,um${\displaystyle u_1,\cdots ,u_m}$ are linear independent vectors in U1${\displaystyle U_1}$ and U2${\displaystyle U_2}$, we extend them to basis of Ui${\displaystyle U_i}$. For example, u1,⋯,um,v1,⋯vn${\displaystyle u_1,\cdots ,u_m,v_1,\cdots v_n}$ is a basis of U1${\displaystyle U_1}$, u1,⋯,um,w1,⋯wk${\displaystyle u_1,\cdots ,u_m,w_1,\cdots w_k}$ is a basis of U2${\displaystyle U_2}$. Now we will prove that u1,⋯,um,v1,⋯,vn,w1,⋯,wk${\displaystyle u_1,\cdots ,u_m,v_1,\cdots ,v_n,w_1,\cdots ,w_k}$ is a basis of U1+U2${\displaystyle U_1+U_2}$.★ First, it's easy to show that u1,⋯,um,v1,⋯,vn,w1,⋯,wk${\displaystyle u_1,\cdots ,u_m,v_1,\cdots ,v_n,w_1,\cdots ,w_k}$ is a spanning list.★ Then, if v1${\displaystyle v_1}$ is linearly dependent with u1,⋯,um,w1,⋯,wk${\displaystyle u_1,\cdots ,u_m,w_1,\cdots ,w_k}$, we can represent v1${\displaystyle v_1}$ with this list. According to the addition reservation, v1${\displaystyle v_1}$ is a vector of U2${\displaystyle U_2}$. Besides, it's a vector of U1${\displaystyle U_1}$, so it's in the intersection U1∩U2${\displaystyle U_1\cap U_2}$, which means it should be represented by the basis list u1,⋯,um${\displaystyle u_1,\cdots ,u_m}$. However, we supposed that u1,⋯,um,v1,⋯vn${\displaystyle u_1,\cdots ,u_m,v_1,\cdots v_n}$ is a basis of U1${\displaystyle U_1}$, v1${\displaystyle v_1}$ cannot be represented by u1,⋯,um${\displaystyle u_1,\cdots ,u_m}$. So v1${\displaystyle v_1}$ must be linear independent with u1,⋯,um,w1,⋯,wk${\displaystyle u_1,\cdots ,u_m,w_1,\cdots ,w_k}$★ So u1,⋯,um,v1,⋯,vn,w1,⋯,wk${\displaystyle u_1,\cdots ,u_m,v_1,\cdots ,v_n,w_1,\cdots ,w_k}$ is a linearly independent list□${\displaystyle }$

Chapter3. Linear MapsHere comes something!★ Fundamental Theorem of Linear Maps★ the matrix of a linear map with respect to given bases★ isomorphic vector spaces★ product spaces★ quotient spaces★ the dual space of a vector space and the dual of a linear map3.A The Vector Space of Linear MapsDefinition and Examples of Linear Maps Now we are ready for one of the key definitions in linear algebra.

Definition31linear map

A linear map from V${\displaystyle V}$ to W${\displaystyle W}$ is a function T:V→W${\displaystyle T:V\to W}$ with the following properties:★ additivityT(u+v)=Tu+Tv, for all u,v∈V ;$$T(u+v)=Tu+Tv,\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}u,v\in V;$$★ homogeneityT(𝜆v)=𝜆T(v), for all 𝜆∈F and all v∈V .$$T(𝜆v)=𝜆T(v),\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}𝜆\in \mathbb{F}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{l}\mathrm{l}v\in V.$$The set of all linear maps from V${\displaystyle V}$ to W${\displaystyle W}$ is denoted L(V,W)${\displaystyle \mathcal{L}(V,W)}$.

More information

★ Some mathematicians use the term linear transform the same as linear map★ For linear maps we often use the notation Tv${\displaystyle Tv}$ as well as the more standard functional notation T(v)${\displaystyle T(v)}$

Example31.1Some linear maps

★ zero map★ identity map★ differential★ integral★ multiplication by x2${\displaystyle x^2}$: T∈L(P(R),P(R)), (Tp)(x):=x2p(x)${\displaystyle T\in \mathcal{L}(\mathcal{P}(\mathbb{R}),\mathcal{P}(\mathbb{R})),(Tp)(x):=x^{2}p(x)}$★ backward shift: T∈L(F∞,F∞), T(x1,x2,x3,⋯):=(x2,x3,⋯)${\displaystyle T\in \mathcal{L}({\mathbb{F}}^{\infty },{\mathbb{F}}^{\infty }),T(x_1,x_2,x_3,\cdots ):=(x_2,x_3,\cdots )}$

The existence part of the next results means we can find a linear map that takes on whatever values we wish on the vectors in a basis. The uniqueness part means that a linear map is completely determined by its values on a basis.

Theorem32Linear maps and basis of domain

Suppose v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$ and w1,⋯,wn∈W${\displaystyle w_1,\cdots ,w_n\in W}$. Then there exists a unique linear map T:V→W${\displaystyle T:V\to W}$ such thatTvj=wj$$T{v}_j=w_j$$for each j=1,⋯,n.${\displaystyle j=1,\cdots ,n.}$Proof We can define a linear map that maps each vj${\displaystyle v_j}$ to wj${\displaystyle w_j}$:T∈L(V,W), T(a1v1+⋯+anvn):=a1w1+⋯anwn.$$T\in \mathcal{L}(V,W),T(a_1{v}_1+\cdots +a_n{v}_n):=a_1{w}_1+\cdots a_n{w}_n.$$And it's easy to show that T${\displaystyle T}$ is a linear map.If there exist another S${\displaystyle S}$ which can map each vj${\displaystyle v_j}$ to wj${\displaystyle w_j}$, then for every vector in the space, two maps must be equal. So actually, there is only one map.

Algebraic Operations on L(V,W)${\displaystyle \mathcal{L}(V,W)}$We begin by defining addition and scalar multiplication on L(V,W)${\displaystyle \mathcal{L}(V,W)}$

Definition33addition and scalar multiplication on L(V,W)${\displaystyle \mathcal{L}(V,W)}$

Suppose S,T∈L(V,W)${\displaystyle S,T\in \mathcal{L}(V,W)}$ and 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$. The sum S+T${\displaystyle S+T}$ and the product 𝜆T${\displaystyle 𝜆T}$ are the linear maps from V ${\displaystyle V}$ to W${\displaystyle W}$ defined by(S+T)(v):=Sv+Tv$$(S+T)(v):=Sv+Tv$$(𝜆T)(v):=𝜆(Tv)$$(𝜆T)(v):=𝜆(Tv)$$

The next result should not be a surprise.

Definition34L(V,W)${\displaystyle \mathcal{L}(V,W)}$ is a vector space

With the operations defined in Definition "addition and scalar multiplication"

Usually it makes no sense to multiply together two elements of a vector space, but for some pairs of linear maps a useful product exists. We will need a third vector space, so for the rest of this section suppose U${\displaystyle U}$ is a vector space over F${\displaystyle \mathbb{F}}$

Definition35Product of Linear Maps

If T∈L(U,V)${\displaystyle T\in \mathcal{L}(U,V)}$ and S∈L(V,W)${\displaystyle S\in \mathcal{L}(V,W)}$, then the product ST∈L(U,W)${\displaystyle ST\in \mathcal{L}(U,W)}$ is defined by(ST)(u):=S(Tu).$$(ST)(u):=S(Tu).$$It's just a notation for linear mapping twice. It's just the usual composition S∘T${\displaystyle S\circ T}$ of two functions.Note that ST${\displaystyle ST}$ is defined only when T${\displaystyle T}$ maps into the domain of S${\displaystyle S}$

3.B Null Spaces and RangesNull Spaces and InjectivityIn this section we will learn about two subspaces that are intimately connected with each linear map.

Definition36null space null T${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$

For T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$, the null space of T${\displaystyle T}$, denoted null T${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$, is the subset of V${\displaystyle V}$ consisting of those vectors that T${\displaystyle T}$ maps to 0${\displaystyle 0}$:null T={v∈V: Tv=0}$$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T=\{v\in V:Tv=0\}$$

Example36.1null space

★ zero map T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$, then null T=V${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T=V}$★ Suppose 𝜑∈L(C3,C)${\displaystyle 𝜑\in \mathcal{L}({\mathbb{C}}^3,\mathbb{C})}$ is defined by 𝜑(z1,z2,z3)=z1+2z2+3z3${\displaystyle 𝜑(z_1,z_2,z_3)=z_1+2z_2+3z_3}$. Then null 𝜑={(z1,z2,z3)∈C3: z1+2z2+3z3=0}$$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}𝜑=\{(z_1,z_2,z_3)\in {\mathbb{C}}^3:z_1+2z_2+3z_3=0\}$$

Some mathematicians use the term kernel instead of null space.

Theorem37The null space is a subspace

ProofAddition and scalar multiplication is closed because of the definition of linear map.□${\displaystyle }$

As we will soon see, for a linear map the next definition is closely connected to the null space.

Definition38injective

A function T${\displaystyle T}$ is called injective if Tu=Tv${\displaystyle Tu=Tv}$ implies u=v${\displaystyle u=v}$, which means 单射

Theorem39The null space and injectivity

Let T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Then T${\displaystyle T}$ is injective if and only if null T={0}${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T=\{0\}}$.

Range and Surjectivity

Definition40Range

For T${\displaystyle T}$ a function from V${\displaystyle V}$ to W${\displaystyle W}$, the range of T${\displaystyle T}$ is the subset of W${\displaystyle W}$ consisting of those vectors that are of the form Tv${\displaystyle Tv}$ for some v∈V${\displaystyle v\in V}$:range T={Tv:v∈V}.$$\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T=\{Tv:v\in V\}.$$Recall that V${\displaystyle V}$ is called domain here.

Also called image

Theorem41The range is a subspace

ProofAddition and scalar multiplication is closed because of the definition of linear map.□${\displaystyle }$

Definition42surjective

A function T${\displaystyle T}$ is called surjective if its range equals W${\displaystyle W}$.

Fundamental Theorem of Linear MapsThe next result is so important that it gets a dramatic name.

Theorem43Fundamental Theorem of Linear Maps

Suppose V${\displaystyle V}$ is finite-dimensional and T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Then range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ is finite-dimensional and dim V=dim null T+dim range T.$$\mathrm{d}\mathrm{i}\mathrm{m}V=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T+\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T.$$Proof Suppose V${\displaystyle V}$ has a m${\displaystyle m}$ length basis v1,v2,⋯,vm${\displaystyle v_1,v_2,\cdots ,v_m}$. When it is mapped into another space, they're changed intowj=Tvj$$w_j=T{v}_j$$It's clearly that wj${\displaystyle w_j}$ spans range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$. Thus we can remove some of the wj${\displaystyle w_j}$ until it remains a basis of range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$. Suppose we classify wj${\displaystyle w_j}$ into two lists: w1,⋯,wk${\displaystyle w_1,\cdots ,w_k}$ and wk+1,⋯,wm${\displaystyle w_{k+1},\cdots ,w_m}$, each wj${\displaystyle w_j}$ in the latter list is linearly dependent with the former. In other words, w1,⋯,wk${\displaystyle w_1,\cdots ,w_k}$ is a basis and range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ has dimension k${\displaystyle k}$. Now I will prove the length n=m-k${\displaystyle n=m-k}$, of list wk+1,⋯,wm${\displaystyle w_{k+1},\cdots ,w_m}$, is the dimension of null T${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$. Supposewj=k∑iaiwi , for j⩾k+1$$w_j=\sum _i^k{a}_i{w}_i,\mathrm{f}\mathrm{o}\mathrm{r}j⩾k+1$$It means Tvj=k∑iaiT(vi)${\displaystyle T{v}_j=\sum _i^k{a}_{i}T(v_i)}$ as well. Thus: let uj=vj-k∑iaivi${\displaystyle u_j=v_j-\sum _i^k{a}_i{v}_i}$, T(uj)=0${\displaystyle T(u_j)=0}$. For every vector in null spaceT(m∑i=1aivi)=0,$$T\left(\sum _{i=1}^m{a}_i{v}_i\right)=0,$$use uj${\displaystyle u_j}$ to replace vj${\displaystyle v_j}$ for j⩾k+1${\displaystyle j⩾k+1}$. There will be more terms of vi${\displaystyle v_i}$ for i⩽k${\displaystyle i⩽k}$. So we collect the coefficients and writeT(k∑i=1a'ivi+m∑j=k+1uj)=0$$T(\sum _{i=1}^k{a}_i^{\text{'}}{v}_i+\sum _{j=k+1}^m{u}_j)=0$$Because the latter term naturally equals 0${\displaystyle 0}$, k∑i=1T(a'ivi)=k∑i=1a'iwi=0$$\sum _{i=1}^{k}T(a{\text{'}}_i{v}_i)=\sum _{i=1}^{k}a{\text{'}}_i{w}_i=0$$So a'i=0${\displaystyle a{\text{'}}_i=0}$. Therefore, for every vector in null space , we can use list uj${\displaystyle u_j}$ to represent it, which means list uj${\displaystyle u_j}$ is a span. Besides, it can be proved that uj${\displaystyle u_j}$ is linearly independent——So the dimension of null space is n=m-k${\displaystyle n=m-k}$!□

Now we can show that no linear map from a finite-dimensional space to a "smaller" space is injective, where "smaller" is measured by dimension

Theorem44A map to a smaller dimensional space is not injective

ProofUse fundamental theorem of linear maps

Theorem45A map to a larger dimensional space is not surjective

ProofSimilarly

The term "homogeneous" below means that all the constant value on the right of the equation system equal 0.

Example45.1Rephrase in terms of a linear map the question of whether a homogeneous system of linear equations has a nonzero solution

The equationT(x1,⋯,xn)=0$$T(x_1,\cdots ,x_n)=0$$means that x1,⋯,xn${\displaystyle x_1,\cdots ,x_n}$ is in the null space. Thus after rephrasing, the question becomes:Whether the linear map is injective.

Theorem46Homogeneous system of linear equations

A homogeneous system of linear equation with more variable than equations has nonzero solutions.ProofMore variable means it's a mapping into smaller space. □${\displaystyle }$

Example46.1Whether an inhomogeneous system of linear equations has no solutions for some choice of the constant terms

If the range has less dimension than vector space W${\displaystyle W}$, then there definitely can be some situation in which the equation has no solution. So the question can be changed as:Whether the linear map is surjective?

Theorem47Inhomogeneous system of linear equations

An inhomogeneous system of linear equation with less variable than equations has no solution for some choice of the constant termsProofMore equation means it's a mapping into bigger space. □${\displaystyle }$

3.C MatricesRepresenting a Linear Map by a Matrix

Definition48Matrix of a linear map, M(T)${\displaystyle \mathcal{M}(T)}$

Suppose T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$ and v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$ and w1,⋯,wm${\displaystyle w_1,\cdots ,w_m}$ is a basis of W${\displaystyle W}$ . The matrix of T ${\displaystyle T}$ with respect to these bases is the m-by-n${\displaystyle m-\mathrm{b}\mathrm{y}-n}$ matrix M(T)${\displaystyle \mathcal{M}(T)}$ whose entries Aj,k${\displaystyle A_{j,k}}$ are defined byTvk=A1,kw1+⋯+Am,kwm$$T{v}_k=A_{1,k}{w}_1+\cdots +A_{m,k}{w}_m$$If the bases are not clear from the context, then the notation M(T(v1,⋯,vn), (w1,⋯,wm))${\displaystyle \mathcal{M}(T(v_1,\cdots ,v_n),(w_1,\cdots ,w_m))}$ is used.

The matrix M(T)${\displaystyle \mathcal{M}(T)}$ of a linear map T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$ depends on the basis v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ of V${\displaystyle V}$ and the basis w1,⋯,wm${\displaystyle w_1,\cdots ,w_m}$ of W${\displaystyle W}$, as well as on T${\displaystyle T}$. However, the bases should be clear from the context, and thus they are often not included in the notation.

Definition49Notation Fm,n${\displaystyle {\mathbb{F}}^{m,n}}$

For m${\displaystyle m}$ and n${\displaystyle n}$ positive integers, the set of all m-by-n${\displaystyle m-\mathrm{b}\mathrm{y}-n}$ matrices with entries in F${\displaystyle \mathbb{F}}$ is denoted by Fm,n${\displaystyle {\mathbb{F}}^{m,n}}$.

Theorem50Linear combination of columns

Suppose A${\displaystyle A}$ is an m-by-n${\displaystyle m-\mathrm{b}\mathrm{y}-n}$ matrix and c=a

c1

⋮

cn

${\displaystyle c=\left(\begin{array}{c}{c}_1\\ ⋮\\ c_n\end{array}\right)}$ is an n-by-1${\displaystyle n-\mathrm{b}\mathrm{y}-1}$ matrix. ThenAc=c1A.,1+⋯+cnA.,1$$Ac=c_1{A}_{.,1}+\cdots +c_n{A}_{.,1}$$In other words, Ac${\displaystyle Ac}$ is a linear combination of the columns of A${\displaystyle A}$, with the scalars that multiply the columns coming from c${\displaystyle c}$.

3.D Invertibility and Isomorphic Vector SpacesInvertible Linear Maps

Definition51invertible, inverse

A linear map T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$ is called invertible if there exists a linear map S∈L(W,V)${\displaystyle S\in \mathcal{L}(W,V)}$ such that ST${\displaystyle ST}$ equals the identity map on V${\displaystyle V}$ and TS${\displaystyle TS}$ equals the identity map on W${\displaystyle W}$ A linear map S∈L(W,V)${\displaystyle S\in \mathcal{L}(W,V)}$ satisfying ST=I${\displaystyle ST=\mathbb{I}}$ and TS=I${\displaystyle TS=\mathbb{I}}$ is called an inverse of T${\displaystyle T}$ (note that the first I${\displaystyle \mathbb{I}}$ is on V${\displaystyle V}$ and the second I${\displaystyle \mathbb{I}}$ is on W${\displaystyle W}$.

Theorem52Inverse is unique

ProofOtherwiseS1=S1I=S1TS2=IS2=S2$$S_1=S_1\mathbb{I}=S_{1}T{S}_2=\mathbb{I}{S}_2=S_2$$Thus S1=S2${\displaystyle S_1=S_2}$ □

Theorem53Invertibility is equivalent to injectivity and surjectivity

A linear map is invertible if and only if it is injective and surjective.Proof ★ "Only if" is clearly to prove. ★ Then if the map is one-to-one, we can just defineS→T(S(w))=w$$S\to T(S(w))=w$$★ Clearly we have T∘S=I${\displaystyle T\circ S=\mathbb{I}}$. Then what about S∘T${\displaystyle S\circ T}$ ? For each vector v${\displaystyle v}$ in T${\displaystyle T}$, surjectivity tells us that we can find w∈S, v=S(w)${\displaystyle w\in S,v=S(w)}$. Thus∵S(T(S(w)))=S(w) S(T(v))=v$$\because S\left(T\right(S(w)\left)\right)=S(w)S(T(v))=v$$★ To complete the proof, we need to show that S${\displaystyle S}$ is linear as well.

Isomorphic Vector SpacesThe next definition captures the idea of two vector spaces that are essentially the same, except for the names of the elements of the vector spaces

Definition54isomorphic, isomorphism

★ An isomorphism is an invertible linear map★ Two vector spaces are called isomorphic if there is an isomorphism from one vector space onto the other one.

More information

The Greek word isos means equal; the Greek word morph means shape. Thus isomorphic literally means equal shape. It means the same time as invertible. Use "isomorphism" when you want to emphasize that the two spaces are essentially the same.

Theorem55Dimension shows whether vector spaces are isomorphic

Two finite-dimensional vector spaces over F${\displaystyle \mathbb{F}}$ are isomorphic if and only if they have the same dimension.Proof First, if vector spaces are isomorphic, then there exists an isomorphic linear map between V${\displaystyle V}$ andW${\displaystyle W}$. Isomorphism asks the null space to be {0}${\displaystyle \{0\}}$. Thus according to the fundamental theorem of linear map, dim W=range M=dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{W}\mathrm{=}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}M=\mathrm{d}\mathrm{i}\mathrm{m}V}$. To complete the proof in another direction, suppose two finite dimensional vector spaces have the same dimension. Thus we can construct a linear map which holds each coefficients of basis label.T(c1v1+⋯+cmvm)=c1w1+⋯+cmwm.$$T(c_1{v}_1+\cdots +c_m{v}_m)=c_1{w}_1+\cdots +c_m{w}_m.$$□

The previous result implies that each finite-dimensional vector space V${\displaystyle V}$ is isomorphic to Fn${\displaystyle {\mathbb{F}}^n}$, where n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. If v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$ and w1,⋯,wm${\displaystyle w_1,\cdots ,w_m}$ is a basis of W${\displaystyle W}$, then for each T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$, we have a matrix M(T)∈Fm,n${\displaystyle \mathcal{M}(T)\in {\mathbb{F}}^{m,n}}$. In other words, once bases have been fixed for V${\displaystyle V}$ and W${\displaystyle W}$, M${\displaystyle \mathcal{M}}$ becomes a function from L(V,W)${\displaystyle \mathcal{L}(V,W)}$ to Fm,n${\displaystyle {\mathbb{F}}^{m,n}}$. Notice M${\displaystyle \mathcal{M}}$ is a linear map, and it's actually invertible, as we will show.

Theorem56L(V,W)${\displaystyle \mathcal{L}(V,W)}$ and Fm,n${\displaystyle {\mathbb{F}}^{m,n}}$ are isomorphic

Suppose v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$ and w1,⋯,wm${\displaystyle w_1,\cdots ,w_m}$ is a basis of W${\displaystyle W}$, Then M${\displaystyle \mathcal{M}}$ is an isomorphism between L(V,W)${\displaystyle \mathcal{L}(V,W)}$ and Fm,n${\displaystyle {\mathbb{F}}^{m,n}}$.

Theorem57dim L(V,W)=(dim V )(dim W )${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathcal{L}(V,W)=(\mathrm{d}\mathrm{i}\mathrm{m}V)(\mathrm{d}\mathrm{i}\mathrm{m}W)}$

M${\displaystyle \mathcal{M}}$ is an isomorphism between L(V,W)${\displaystyle \mathcal{L}(V,W)}$ and Fm,n${\displaystyle {\mathbb{F}}^{m,n}}$. Thus L${\displaystyle \mathcal{L}}$ has the same dimension with Fm,n${\displaystyle {\mathbb{F}}^{m,n}}$ ∴dim Fm,n=mn${\displaystyle \therefore \mathrm{d}\mathrm{i}\mathrm{m}{\mathbb{F}}^{m,n}=mn}$

Linear Maps thought of as Matrix MultiplicationPreviously we defined the matrix of a linear map. Now we define the matrix of a vector.

Definition58matrix of a vector, M(v)${\displaystyle \mathcal{M}(v)}$

Suppose v∈V${\displaystyle v\in V}$ and v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$. For v=n∑icivi${\displaystyle v=\sum _i^n{c}_i{v}_i}$, defineM(v):=a

c1

⋮

cn

$$\mathcal{M}(v):=\left(\begin{array}{c}{c}_1\\ ⋮\\ c_n\end{array}\right)$$

M(v)${\displaystyle \mathcal{M}(v)}$ is an isomorphism of V${\displaystyle V}$ onto Fn,1${\displaystyle {\mathbb{F}}^{n,1}}$.

Theorem59M(T).,k=M(vk)${\displaystyle \mathcal{M}(T{)}_{.,k}=\mathcal{M}(v_k)}$ (? It may should be M(T).,k=M(T(vk))${\displaystyle \mathcal{M}(T{)}_{.,k}=\mathcal{M}(T(v_k))}$

Suppose T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$ and v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$ and w1,⋯,wm${\displaystyle w_1,\cdots ,w_m}$ is a basis of W${\displaystyle W}$. M(T)${\displaystyle \mathcal{M}(T)}$ then is defined by ci= ∑jMijaj${\displaystyle c_i=\sum _j^{}{\mathcal{M}}_{ij}{a}_j}$. So the k${\displaystyle k}$th column is M.,k${\displaystyle {\mathcal{M}}_{.,k}}$, and Tvk=Mij⋅𝛿j-k=M.,k${\displaystyle T{v}_k=M_{ij}\cdot {𝛿}_{j-k}={\mathcal{M}}_{.,k}}$.

It means, for some linear map, the kth column of the matrix equals the matrix of " the map of the kth basis element"

Theorem60Linear maps act like matrix multiplication

Suppose v=a1v1+⋯+anvn${\displaystyle v=a_1{v}_1+\cdots +a_n{v}_n}$a

M(Tv)	=a1M(Tv1)+a2M(Tv2)+⋯+anM(Tvn)
	=a1M(T).,1+⋯+anM(T).,n
	=M(T)M(v)

$$\begin{array}{rl}\mathcal{M}(Tv)& =a_1\mathcal{M}(T{v}_1)+a_2\mathcal{M}(T{v}_2)+\cdots +a_n\mathcal{M}(T{v}_n)\\ & =a_1\mathcal{M}(T{)}_{.,1}+\cdots +a_n\mathcal{M}(T{)}_{.,n}\\ & =\mathcal{M}(T)\mathcal{M}(v)\end{array}$$

Because the result above allows us to think (via isomorphism) of each linear map as multiplication of Fn,1${\displaystyle {\mathbb{F}}^{n,1}}$ by some matrix A${\displaystyle A}$, keep in mind that the specific matrix A${\displaystyle A}$ depends not only on the linear map but also on the choice of bases. One of the themes of many of the most important results in later chapters will be the choice of a basis that makes the matrix A as simple as possible. Operators

Definition61operator, L(V)${\displaystyle \mathcal{L}(V)}$

A linear map from a vector space to itself is called an operator.L(V)=L(V,V)$$\mathcal{L}(V)=\mathcal{L}(V,V)$$

Recall that injective and surjective implies invertibility. However, I'll show that in finite-dimensional vector space, surjective is equivalent to injective in operator.

Theorem62injectivity is equivalent to surjectivity in finite-dimensional operator

ProofUse fundamental theorem of linear maps. injectivity means null V={0}${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}V=\{0\}}$, thusdim range M=dim V-0=dim V$$\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}M=\mathrm{d}\mathrm{i}\mathrm{m}V-0=\mathrm{d}\mathrm{i}\mathrm{m}V$$vice versa. □

3.E Products and Quotients of Vector SpacesProducts of Vector SpacesAs usual when dealing with more than one vector spaces, all the vector spaces in use should be over the same field.

Definition63product of vector spaces

Suppose V1,⋯,Vm${\displaystyle V_1,\cdots ,V_m}$ are vector spaces over F${\displaystyle \mathbb{F}}$.★ The product V1×⋯×Vm${\displaystyle V_1\times \cdots \times V_m}$ is defined byV1×⋯×Vm={(v1,⋯,vm):v1∈V1,⋯,vm∈Vm}$$V_1\times \cdots \times V_m=\{(v_1,\cdots ,v_m):v_1\in V_1,\cdots ,v_m\in V_m\}$$★ Addition on V1×⋯×Vm${\displaystyle V_1\times \cdots \times V_m}$ is defined by(u1,⋯,um)+(v1,⋯,vm)=(u1+v1,⋯,um+vm)$$(u_1,\cdots ,u_m)+(v_1,\cdots ,v_m)=(u_1+v_1,\cdots ,u_m+v_m)$$★ Scalar multiplication on V1×⋯×Vm${\displaystyle V_1\times \cdots \times V_m}$ is defined similarly

Example63.1R2×R3${\displaystyle {\mathbb{R}}^2\times {\mathbb{R}}^3}$ is isomorphic to R5${\displaystyle {\mathbb{R}}^5}$

In this case, the isomorphism is so natural that we should think of it as a relabeling. Some people would even informally say that R2×R3=R5${\displaystyle {\mathbb{R}}^2\times {\mathbb{R}}^3={\mathbb{R}}^5}$, which is not technically correct but which captures the spirit of identification via relabeling.

Example63.2Find a basis of P2(R)×R${\displaystyle {\mathcal{P}}_2(\mathbb{R})\times \mathbb{R}}$

Recall that product is kind of like "sum" in dimension. The basis is(1,(0,0));(x,(0,0));(x2,(0,0));(0,(1,0));(0,(0,1))$$(1,(0,0));(x,(0,0));(x^2,(0,0));(0,(1,0));(0,(0,1))$$

Theorem64Dimension of a product is the sum of dimensions

dim(V1×⋯×Vm)=dim V1+⋯+dim Vm$$\mathrm{d}\mathrm{i}\mathrm{m}(V_1\times \cdots \times V_m)=\mathrm{d}\mathrm{i}\mathrm{m}{V}_1+\cdots +\mathrm{d}\mathrm{i}\mathrm{m}{V}_m$$

Products and Direct SumsIn the next result, the map Γ${\displaystyle \Gamma }$ is surjective by the definition of U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$. Thus the last word in the result below could be changed from "injective" to "invertible".

Theorem65Products and direct sums

Suppose that U1,⋯,Um${\displaystyle U_1,\cdots ,U_m}$ are subspaces of V${\displaystyle V}$. Define a linear map Γ${\displaystyle \Gamma }$: U1×⋯×Um→U1+⋯+Um${\displaystyle U_1\times \cdots \times U_m\to U_1+\cdots +U_m}$ by𝛤(u1,⋯,um)=u1+⋯+um.$$𝛤(u_1,\cdots ,u_m)=u_1+\cdots +u_m.$$Then U1+⋯+Um${\displaystyle U_1+\cdots +U_m}$ is a direct sum if and only if Γ${\displaystyle \Gamma }$ is injective.Proof If Γ${\displaystyle \Gamma }$ is injective, then for each u1+⋯+um${\displaystyle u_1+\cdots +u_m}$, the original u1,⋯,um${\displaystyle u_1,\cdots ,u_m}$ is unique, which means that is a direct sum. vice versa. □

Theorem66A sum is a direct sum if and only if dimensions add up

Proof The products has dimension dimV1+dimV2+⋯${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}{V}_1+\mathrm{d}\mathrm{i}\mathrm{m}{V}_2+\cdots }$. It's a direct sum if and only if Γ${\displaystyle \Gamma }$ is invertible, which is a linear map between finite-dimensional vector space. Thus only if the range equals dimension of domain, can the map be isomorphism.

Quotients of Vector SpaceWe begin our approach to quotient spaces by defining the sum of a vector and a subspace.

Definition67v+U${\displaystyle v+U}$

Suppose v∈V${\displaystyle v\in V}$ and U${\displaystyle U}$ is a subspaces of V${\displaystyle V}$. Then v+U${\displaystyle v+U}$ is the subset of V${\displaystyle V}$ defined byv+U={v+u:u∈U}.$$v+U=\{v+u:u\in U\}.$$

Definition68affine subset, parallel

★ An affine subset is a subset of V${\displaystyle V}$ of the form v+U${\displaystyle v+U}$ for some v∈V${\displaystyle v\in V}$ and some subspace U${\displaystyle U}$ of V${\displaystyle V}$.★ For v∈V${\displaystyle v\in V}$ and U${\displaystyle U}$ a subspace of V${\displaystyle V}$, the affine subset v+U${\displaystyle v+U}$ is said to be parallel to U${\displaystyle U}$

Definition69quotient space, V/U${\displaystyle V/U}$

Suppose U is a subspace of V${\displaystyle V}$. Then the quotient space V/U${\displaystyle V/U}$ is the set of all affine subsets of V${\displaystyle V}$ parallel to U${\displaystyle U}$. In other words,V/U={v+U:v∈V}.$$V/U=\{v+U:v\in V\}.$$

Our next goal is to make V/U${\displaystyle V/U}$ into a vector space. To do this, we will need the following result.

Theorem70Two affine subsets parallel to U${\displaystyle U}$ are equal or disjoint

Suppose U${\displaystyle U}$ is a subspace of V ${\displaystyle V}$and v,w∈V${\displaystyle v,w\in V}$. Then the following are equivalent:(2)a

(1)			v-w∈U
(2)			v+U=w+U
(3)			(v+U)∩(w+U)≠∅

$$\begin{array}{rlrl}(1)& & & v-w\in U\\ (2)& & & v+U=w+U\\ (3)& & & (v+U)\cap (w+U)\ne \varnothing \end{array}$$Proof if (1) holds, so v+u=w+(v-w+u)∈w+U${\displaystyle v+u=w+(v-w+u)\in w+U}$, vice versa. So (2) holds. If (3) holds, so (v+U)∩(w+U)≠∅${\displaystyle (v+U)\cap (w+U)\ne \varnothing }$. Thus there exist u1,u2∈U${\displaystyle u_1,u_2\in U}$ such thatv1+u1=w+u2.$$v_1+u_1=w+u_2.$$Thus v-w=u2-u1∈U${\displaystyle v-w=u_2-u_1\in U}$

Now we can define addition and scalar multiplication on V/U${\displaystyle V/U}$

Definition71addition and scalar multiplication on V/U${\displaystyle V/U}$

Defined as followed:(v+U)+(w+U):=(v+w)+U𝜆(v+U):=𝜆v+U$$\begin{array}{c}(v+U)+(w+U):=(v+w)+U\\ 𝜆(v+U):=𝜆v+U\end{array}$$

Theorem72Quotient Space is a Vector Space

Suppose U${\displaystyle U}$ is a subspace of V${\displaystyle V}$. Then V/U${\displaystyle V/U}$, with the operations of addition and scalar multiplication as defined above, is a vector space.The additive identity of V/U${\displaystyle V/U}$ is 0+U${\displaystyle 0+U}$ and that the additive inverse of v+U${\displaystyle v+U}$ is (-v)+U${\displaystyle (-v)+U}$.

VU, a subspace of VV/U

Fig1:Quotient Space

The next concept will gives us an easy way to compute the dimension of V/U${\displaystyle V/U}$.

Definition73quotient map, 𝜋${\displaystyle 𝜋}$

Suppose U${\displaystyle U}$ is a subspace of V${\displaystyle V}$. The quotient map 𝜋${\displaystyle 𝜋}$ is the linear map 𝜋:V→V/U${\displaystyle 𝜋:V\to V/U}$ defined by𝜋(v):=v+U$$𝜋(v):=v+U$$for v∈V${\displaystyle v\in V}$. 𝜋${\displaystyle 𝜋}$ should depends on U, V${\displaystyle U,V}$.

Definition74Dimension of a quotient space

Suppose V${\displaystyle V}$ is finite-dimensional and U${\displaystyle U}$ is a subspace of V${\displaystyle V}$. Thendim V/U:=dimV-dimU$$\mathrm{d}\mathrm{i}\mathrm{m}V/U:=\mathrm{d}\mathrm{i}\mathrm{m}V-\mathrm{d}\mathrm{i}\mathrm{m}U$$Proof Use the fundamental theorem of linear map. Consider map 𝜋${\displaystyle 𝜋}$, its null space is U${\displaystyle U}$ itself. Then it gives the desired result.

Each linear map T${\displaystyle T}$ on V${\displaystyle V}$ induces a linear map T${\displaystyle \tilde{T}}$ on V/(null T)${\displaystyle V/(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T)}$, which we now define.

Definition75T${\displaystyle \tilde{T}}$

Suppose T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Define T: V/(null T)→W${\displaystyle \tilde{T}:V/(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T)\to W}$ byT(v+null T)=Tv.$$\tilde{T}(v+nullT)=Tv.$$It means that a group of vectors parallel to null has the same result.

To show that the definition of T ${\displaystyle \tilde{T}}$makes sense, suppose u,v∈V${\displaystyle u,v\in V}$ are such that u+null T=v+null T${\displaystyle u+\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T=v+\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$. Then according to (2), there must be u-v∈null T${\displaystyle u-v\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$. Thus T(u-v)=0${\displaystyle T(u-v)=0}$. Hence Tu=Tv${\displaystyle Tu=Tv}$. The definition of T ${\displaystyle \tilde{T}}$indeed make sense.

Definition76Null space and range of T${\displaystyle \tilde{T}}$

Suppose T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Then(1) T ${\displaystyle \tilde{T}}$is a linear map from V/(null T)${\displaystyle V/(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T)}$ to W${\displaystyle W}$;(2) T ${\displaystyle \tilde{T}}$is injective;(3) range T=range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\tilde{T}=\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$;(4) V/(null T)${\displaystyle V/(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T)}$ is isomorphic to range T${\displaystyle T}$. Proof (1) omit(2) We need to prove that null T={0}${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}\tilde{T}=\{0\}}$. Recall that the domain of T ${\displaystyle \tilde{T}}$is a quotient space. So for different v∈V: Tv=0${\displaystyle v\in V:Tv=0}$, they all in the null space v∈null T${\displaystyle v\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$. Thus vi+null T${\displaystyle v_i+\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$ is a same thing.

3.F DualityThe Dual Space and the Dual MapLinear maps into the scalar field F${\displaystyle \mathbb{F}}$ play a special role in linear algebra, and thus they get a special name:

Definition77linear functional

A linear functional on V${\displaystyle V}$ is a linear map from V${\displaystyle V}$ to F${\displaystyle \mathbb{F}}$. In other words, a linear functional is an element of L(V,F)${\displaystyle \mathcal{L}(V,\mathbb{F})}$.

Example77.1linear functionals

★ Define 𝜑: R3→R${\displaystyle 𝜑:{\mathbb{R}}^3\to \mathbb{R}}$ by 𝜑(x,y,z):=4x-5y+2z${\displaystyle 𝜑(x,y,z):=4x-5y+2z}$. Then 𝜑${\displaystyle 𝜑}$ is a linear functional on R3${\displaystyle {\mathbb{R}}^3}$.★ Fix (c1,⋯,cn)∈Fn${\displaystyle (c_1,\cdots ,c_n)\in {\mathbb{F}}^n}$. Define 𝜑: Fn→F${\displaystyle 𝜑:{\mathbb{F}}^n\to \mathbb{F}}$ by𝜑(x1,⋯,xn)= ∑icixi$$𝜑(x_1,\cdots ,x_n)=\sum _i^{}{c}_i{x}_i$$★ Define 𝜑: P(R)→R${\displaystyle 𝜑:\mathcal{P}(\mathbb{R})\to \mathbb{R}}$ by 𝜑(p)=1∫0p(x)dx${\displaystyle 𝜑(p)=\underset{0}{\overset{1}{\int }}p(x)\mathrm{d}x}$. ★ Define 𝜑: P(R)→R${\displaystyle 𝜑:\mathcal{P}(\mathbb{R})\to \mathbb{R}}$ by 𝜑(p)=3p''(5)+7p(4)${\displaystyle 𝜑(p)=3p\text{'}\text{'}(5)+7p(4)}$.

The vector space L(V,F)${\displaystyle \mathcal{L}(V,\mathbb{F})}$ also gets a special name and special notation:

Definition78dual space, V'${\displaystyle V\text{'}}$

The dual space of V${\displaystyle V}$, denoted V'${\displaystyle V\text{'}}$, is the vector space of all linear functionals on V${\displaystyle V}$. In other words, V'=L(V,F).$$V\text{'}=\mathcal{L}(V,\mathbb{F}).$$

Theorem79dim V'=dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}V}$

Suppose V${\displaystyle V}$ is finite-dimensional. Then dim V'=dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}V}$Proofdim L=dim V×dimW${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathcal{L}=\mathrm{d}\mathrm{i}\mathrm{m}V\times \mathrm{d}\mathrm{i}\mathrm{m}W}$

Definition80dual basis

If v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$, then the dual basis of v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is the list 𝜑1,⋯,𝜑n${\displaystyle {𝜑}_1,\cdots ,{𝜑}_n}$ of elements of V'${\displaystyle V\text{'}}$, where each 𝜑j${\displaystyle {𝜑}_j}$ is the linear functional on V${\displaystyle V}$ such that𝜑j(vk)=a

1		if k=j,
0		if k≠j.

$${𝜑}_j(v_k)=\{\begin{array}{ll}1& \mathrm{i}\mathrm{f}k=j,\\ 0& \mathrm{i}\mathrm{f}k\ne j.\end{array}$$

The next result shows that the dual basis is indeed a basis.

Theorem81Dual basis is a basis of the dual space

ProofWe only need to prove that each element in the basis is linearly independent, which is obvious.

Definition82dual map, T'${\displaystyle T\text{'}}$ If T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$, then the dual map of T${\displaystyle T}$ is the linear map T'∈L(W',V')${\displaystyle T\text{'}\in \mathcal{L}(W\text{'},V\text{'})}$ defined by T'(𝜑)=𝜑∘T${\displaystyle T\text{'}(𝜑)=𝜑\circ T}$ for 𝜑∈W'${\displaystyle 𝜑\in W\text{'}}$. Recall that, 𝜑∈W'${\displaystyle 𝜑\in W\text{'}}$, so the dual map itself is a linear functional. It can operate on a linear functional in dual space, find a corresponding linear functional in V'${\displaystyle V\text{'}}$, which brings any v${\displaystyle v}$ into 𝜑∘T(v)${\displaystyle 𝜑\circ T(v)}$.

VWV'W'TT'linear functional Fig2:dual map

Example82.1dual map

It's like mapping twice. When you map v∈V${\displaystyle v\in V}$ into W${\displaystyle W}$, you choose a basis in W${\displaystyle W}$ to define a linear functional. After doing this, you want to define a linear functional in space V${\displaystyle V}$ as well. And you want to make sure that these to linear functional has some connection. For example, in this situation, if you choose a good basis and make the T${\displaystyle T}$ map between V ${\displaystyle V}$and W${\displaystyle W}$ very simple: aivi→aiwi${\displaystyle a_i{v}_i\to a_i{w}_i}$. Then this dual map will create some linear functionals in V': v'=𝜑s∘T.${\displaystyle V\text{'}:v\text{'}={𝜑}_s\circ T.}$Then let v'${\displaystyle v\text{'}}$ map a vector from V${\displaystyle V}$ into F${\displaystyle \mathbb{F}}$: v' (aivi)=𝜑∘T(aivi)=𝜑∘aiwi=ai${\displaystyle v\text{'}(a_i{v}_i)=𝜑\circ T(a_i{v}_i)=𝜑\circ a_i{w}_i=a_i}$

Example82.2an image of dual map

a

a x1 x2 x3	→	a a11 a12 a13 a21 a22 a23 a x1 x2 x3	→	a y1 y2
dual↓		dual map		dual↓
a x4 x5 x6	←	a b11 b21 b12 b22 b13 b23 a y3 y4	←	a y3 y4

$$\begin{array}{rlrlr}\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)& \to & \left(\begin{array}{ccc}{a}_{11}& a_{12}& a_{13}\\ a_{21}& a_{22}& a_{23}\end{array}\right)\left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)& \to & \left(\begin{array}{c}{y}_1\\ y_2\end{array}\right)\\ dual\downarrow & & dualmap& & dual\downarrow \\ \left(\begin{array}{ccc}{x}_4& x_5& x_6\end{array}\right)& \leftarrow & \left(\begin{array}{cc}{b}_{11}& b_{21}\\ b_{12}& b_{22}\\ b_{13}& b_{23}\end{array}\right)\left(\begin{array}{cc}{y}_3& y_4\end{array}\right)& \leftarrow & \left(\begin{array}{cc}{y}_3& y_4\end{array}\right)\end{array}$$

Theorem83Algebraic properties of dual maps

★ (S+T)'=S'+T' for all S,T∈ L(V,W)$$(S+T)\text{'}=S\text{'}+T\text{'}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}S,T\in \mathcal{L}(V,W)$$★ (𝜆T)'=𝜆T' for all T∈L(V,W)$$(𝜆T)\text{'}=𝜆T\text{'}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}T\in \mathcal{L}(V,W)$$★ (ST)'=T'S' for all T∈L(U,V) and all S∈L(V,W)$$(ST)\text{'}=T\text{'}S\text{'}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}T\in \mathcal{L}(U,V)\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{l}\mathrm{l}S\in \mathcal{L}(V,W)$$Proof ★ (S+T)'𝜑=𝜑∘(S+T)=𝜑∘S+𝜑∘T=S'𝜑+T'𝜑$$(S+T)\text{'}𝜑=𝜑\circ (S+T)=𝜑\circ S+𝜑\circ T=S\text{'}𝜑+T\text{'}𝜑$$★ Suppose 𝜑∈W'${\displaystyle 𝜑\in W\text{'}}$, then(ST)'𝜑=𝜑∘(ST)=(𝜑∘S)∘T$$(ST)\text{'}𝜑=𝜑\circ (ST)=(𝜑\circ S)\circ T$$the first term is a linear functional whose domain is T${\displaystyle T}$. Thus according to the dual map definition=T'(𝜑∘S)=T'S'(𝜑)□$$=T\text{'}(𝜑\circ S)=T\text{'}S\text{'}(𝜑)$$

The null Space and Range of the Dual of a Linear MapOur goal in this subsection is to describe null T'${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\text{'}}$ and range T'${\displaystyle T\text{'}}$ in terms of range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ and null T${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$.

Definition84annihilator, U0${\displaystyle U^0}$

For U⊂V${\displaystyle U\subset V}$,the annihilator of U${\displaystyle U}$, denoted U0${\displaystyle U^0}$, is defined byU0={𝜑∈V':𝜑(u)=0 for all u∈U}$$U^0=\{𝜑\in V\text{'}:𝜑(u)=0\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}u\in U\}$$

How can a linear map from V${\displaystyle V}$ operate on U${\displaystyle U}$? In my opinion, to be more specific, we can construct a linear map from V'${\displaystyle V\text{'}}$ to U'${\displaystyle U\text{'}}$ to show this:

linear functionalT'TV'U'VUT(u)=u;T'(v')=v'T${\displaystyle T}$ is just an "identity" map from U${\displaystyle U}$ into V${\displaystyle V}$, so is T'${\displaystyle T\text{'}}$. when we talk about U0${\displaystyle U^0}$, we are talking about the linear functional that maps all u∈U${\displaystyle u\in U}$ into 0. Thus it's actually a 0${\displaystyle 0}$ map, whose domain is U${\displaystyle U}$, and it's 0${\displaystyle 0}$ in the dual space U'${\displaystyle U\text{'}}$. Now, we can answer the question -- what's annihilator in fact? It's just the null space of T' !

Example84.1 the annihilator of P(R)${\displaystyle \mathcal{P}(\mathbb{R})}$

Suppose U${\displaystyle U}$ is the subspace of P(R)${\displaystyle \mathcal{P}(\mathbb{R})}$ consisting of all polynomial multiples of x2${\displaystyle x^2}$. If 𝜑${\displaystyle 𝜑}$ is the linear functional on P(R)${\displaystyle \mathcal{P}(\mathbb{R})}$ defined by 𝜑(p)=p'(0)${\displaystyle 𝜑(p)=p\text{'}(0)}$, then 𝜑∈U0.${\displaystyle 𝜑\in U^0.}$

For U⊂V${\displaystyle U\subset V}$, the annihilator U0${\displaystyle U^0}$ is a subset of the dual space V'${\displaystyle V\text{'}}$. Thus U0${\displaystyle U^0}$ depends on the vector space containing U${\displaystyle U}$, so a notation such as U0V${\displaystyle U_V^0}$ would be more precise.

Example84.2

Let e1,e2,⋯,e5${\displaystyle e_1,e_2,\cdots ,e_5}$ denote the standard basis of R5${\displaystyle {\mathbb{R}}^5}$, and let 𝜑1,𝜑2,⋯𝜑5${\displaystyle {𝜑}_1,{𝜑}_2,\cdots {𝜑}_5}$ denote the dual basis of (R5)'.${\displaystyle \left({\mathbb{R}}^5\right)\text{'}.}$ SupposeU=span(e1,e2)={(x1,x2,0,0,0)∈R5:x1,x2∈R}.$$U=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(e_1,e_2)=\{(x_1,x_2,0,0,0)\in {\mathbb{R}}^5:x_1,x_2\in \mathbb{R}\}.$$Show that U0=${\displaystyle U^0=}$span(𝜑3,𝜑4,𝜑5)${\displaystyle ({𝜑}_3,{𝜑}_4,{𝜑}_5)}$Solutionomit.

Theorem85The annihilator is a subspace

Suppose U⊂V.${\displaystyle U\subset V.}$ Then U0${\displaystyle U^0}$ is a subspace of V'${\displaystyle V\text{'}}$.Proof Define the addition and scalar multiplication as normal.Clearly 0∈U0${\displaystyle 0\in U^0}$.

Theorem86Dimension of the annihilator

Suppose V${\displaystyle V}$ is finite-dimensional vector space and U${\displaystyle U}$ is a subspace of V${\displaystyle V}$. Thendim U+dim U0=dim V$$\mathrm{d}\mathrm{i}\mathrm{m}U+\mathrm{d}\mathrm{i}\mathrm{m}{U}^0=\mathrm{d}\mathrm{i}\mathrm{m}V$$Proof

VUU'Informally: For a list of basis of U${\displaystyle U}$, it can be extended to the basis of V${\displaystyle V}$. Thus the dual basis of U${\displaystyle U}$ only has the dimension of the "absent" bases number in U${\displaystyle U}$. Formally: because we can define a linear map from U${\displaystyle U}$ to V${\displaystyle V}$, such that i(u)=u, for u ∈U${\displaystyle i(u)=u,foru\in U}$. Thus i'${\displaystyle i\text{'}}$ is a map from V'${\displaystyle V\text{'}}$ into U'${\displaystyle U\text{'}}$. Using fundamental theorem(3)dim null i'+dim range i'=dim V'=dim V$$\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}i\text{'}+\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}i\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}V\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}V$$and null i'={i'v': i'(v')=0}$$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}i\text{'}=\{i\text{'}v\text{'}:i\text{'}(v\text{'})=0\}$$where i'(v')=0${\displaystyle i\text{'}(v\text{'})=0}$ means 𝜑(u)=0 for all u∈U${\displaystyle 𝜑(u)=0forallu\in U}$. Thus null i'=U0${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}i\text{'}=U^0}$. And the range of i'${\displaystyle i\text{'}}$ is clearly U'${\displaystyle U\text{'}}$, whose dimensional equals dim U${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}U}$. So equ(3) becomesdim U0+dim U=dim V□$$\mathrm{d}\mathrm{i}\mathrm{m}{U}^0+\mathrm{d}\mathrm{i}\mathrm{m}U=\mathrm{d}\mathrm{i}\mathrm{m}V$$

We have show the null space of the T'${\displaystyle T\text{'}}$ from a dual space V'${\displaystyle V\text{'}}$ to the dual space of a subspace of V${\displaystyle V}$, which is U'${\displaystyle U\text{'}}$. Now, what is the null space of a random dual map T'${\displaystyle T\text{'}}$?

Theorem87The null space of T'${\displaystyle T\text{'}}$

Suppose V${\displaystyle V}$ and W${\displaystyle W}$ are finite-dimensional and T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Then★ null T'=(range T)0${\displaystyle T\text{'}=(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T{)}^0}$★ dim null T'=dim null T+dim W-dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T+\mathrm{d}\mathrm{i}\mathrm{m}W-\mathrm{d}\mathrm{i}\mathrm{m}V}$Proof Recall that T'𝜑:=𝜑∘T${\displaystyle T\text{'}𝜑:=𝜑\circ T}$. What's null T'${\displaystyle T\text{'}}$?null T'𝜑(v):=𝜑∘(Tv)=0$$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\text{'}𝜑(v):=𝜑\circ (Tv)=0$$What kind of 𝜑${\displaystyle 𝜑}$ make T(v)=0?${\displaystyle T(v)=0?}$——𝜑∈(Tv)0${\displaystyle 𝜑\in (Tv{)}^0}$. So the first theorem is proved. Once the first theorem is proved, using the previous theorem, we have(4)dim null T'=dim(range T)0=dim W-dim(range T).$$\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T{)}^0=\mathrm{d}\mathrm{i}\mathrm{m}W-\mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T).$$Using the fundamental theorem:dim(range T)=dim V-dim(null T)$$\mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T)=\mathrm{d}\mathrm{i}\mathrm{m}V-\mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T)$$then(5)dim null T'=dim null T+dim W-dim V$$\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T+\mathrm{d}\mathrm{i}\mathrm{m}W-\mathrm{d}\mathrm{i}\mathrm{m}V$$□

Trange T→dim1null T→dim3000T'00a

null T'

=(range T)0→ dim2

a

dim1	=dimW-dim2
	=dimW-dimV+dim3

Fig3:relationships between null T, null T',(range T)0${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T,\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\text{'},(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T{)}^0}$

Theorem88T${\displaystyle T}$ surjective is equivalent to T'${\displaystyle T\text{'}}$ injective

Suppose V${\displaystyle V}$ and W${\displaystyle W}$ are finite-dimensional and T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Then T${\displaystyle T}$ is surjective if and only if T'${\displaystyle T\text{'}}$ is injective.ProofSee equ(4)

Theorem89The range of T'${\displaystyle T\text{'}}$

Suppose V${\displaystyle V}$ and W${\displaystyle W}$ are finite-dimensional and T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Then★ (6)dim range T'=dim range T$$\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T$$★ (7)range T'=(null T)0$$\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}=(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T{)}^0$$Proof dim range T'=dim W-dim null T'${\displaystyle T\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}W-\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\text{'}}$, using equ (5) a

dim range T'	=dim W-(dim null T+dim W-dim V)
	=dim V-dim null T
	=dim range T

$$\begin{array}{rl}\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}& =\mathrm{d}\mathrm{i}\mathrm{m}W-(\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T+\mathrm{d}\mathrm{i}\mathrm{m}W-\mathrm{d}\mathrm{i}\mathrm{m}V)\\ & =\mathrm{d}\mathrm{i}\mathrm{m}V-\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\\ & =\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\end{array}$$ For range T'${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}}$:a

T'(𝜑)	={𝜑∘T: for 𝜑∈W'}
range T'	={𝜓:𝜓=T'𝜑=𝜑∘T, for some 𝜑∈W'}

$$\begin{array}{rl}T\text{'}(𝜑)& =\{𝜑\circ T:for𝜑\in W\text{'}\}\\ \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}& =\{𝜓:𝜓=T\text{'}𝜑=𝜑\circ T,forsome𝜑\in W\text{'}\}\end{array}$$thus for all 𝜓∈T'${\displaystyle 𝜓\in T\text{'}}$, the result they act on v∈V${\displaystyle v\in V}$ is 𝜓v=𝜑∘Tv, for some 𝜑∈W'$$𝜓v=𝜑\circ Tv,forsome𝜑\in W\text{'}$$If v∈null T${\displaystyle v\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$, then𝜓(v)=𝜑∘0=0$$𝜓(v)=𝜑\circ 0=0$$𝜓∈range T'${\displaystyle 𝜓\in \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}}$ can map every vector in null T${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$ into 0${\displaystyle 0}$, thus range T'⊆(null T)0$$\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}\subseteq (\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T{)}^0$$Besides, equ(6) tells us dim rangeT'=dim range T=dim V-dim(null T)=dim(null T)0${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T=\mathrm{d}\mathrm{i}\mathrm{m}V-\mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T)=\mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T{)}^0}$. Thus we have equ(7)□

Theorem90T${\displaystyle T}$ injective is equivalent to T'${\displaystyle T\text{'}}$ surjective

Suppose V${\displaystyle V}$ and W${\displaystyle W}$ are finite-dimensional and T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Then T${\displaystyle T}$ is injective if and only if T'${\displaystyle T\text{'}}$ is surjective.ProofT${\displaystyle T}$ is injective if and only if null T={0}${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T=\{0\}}$, which means(null T)0=V'.$$(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T{)}^0=V\text{'}.$$Using equ(7), it tells us T'${\displaystyle T\text{'}}$ is surjective.

The Matrix of the Dual map of a Linear MapWe now define the transpose of a matrix.

Definition91Transpose

(At)k,j:=Aj,k$$(A^t{)}_{k,j}:=A_{j,k}$$

Theorem92The transpose of the product of matrices

(AC)t=CtAt$$(AC{)}^t=C^t{A}^t$$Proof a

((AC)tk,j)	=(AC)j,k= ∑iAjiCik
	= ∑iAtijCtki= ∑iCtkiAtij
	=(AtCt)k,j

$$\begin{array}{rl}((AC{{)}^t}_{k,j})& =(AC{)}_{j,k}=\sum _i^{}{A}_{ji}{C}_{ik}\\ & =\sum _i^{}{A}_{ij}^t{C}_{ki}^t=\sum _i^{}{C}_{ki}^t{A}_{ij}^t\\ & =(A^t{C}^t{)}_{k,j}\end{array}$$

The setting for the next result is the assumption that we have a basis v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ of V${\displaystyle V}$, along with its dual basis 𝜑1,⋯,𝜑n${\displaystyle {𝜑}_1,\cdots ,{𝜑}_n}$ of V'${\displaystyle V\text{'}}$.

Theorem93The matrix of T'${\displaystyle T\text{'}}$ is the transpose of the matrix of T${\displaystyle T}$

Suppose T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$, M(T')=(M(T))t$$\mathcal{M}(T\text{'})=(\mathcal{M}(T){)}^t$$Proof First, recall the meaning of a matrix times a matrix of vector:M(T')M(w')i:=the ith coefficient of the map of w'$$\mathcal{M}(T\text{'})\mathcal{M}(w\text{'}{)}_i:=\mathrm{t}\mathrm{h}\mathrm{e}{i}^{\mathrm{t}\mathrm{h}}\mathrm{c}\mathrm{o}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{m}\mathrm{a}\mathrm{p}\mathrm{o}\mathrm{f}\mathrm{w}\mathrm{\text{'}}$$BecauseT'w'= ∑iT'(wi𝜑i)= ∑iwi𝜑i∘T(T'w'(v))= ∑iwi𝜑i∘T(v)$$\begin{array}{c}T\text{'}w\text{'}=\sum _i^{}T\text{'}(w_i{𝜑}_i)=\sum _i^{}{w}_i{𝜑}_i\circ T\\ (T\text{'}w\text{'}(v))=\sum _i^{}{w}_i{𝜑}_i\circ T(v)\end{array}$$Because 𝜑i∘T(vj)=( ∑jTijvj)${\displaystyle {𝜑}_i\circ T(v_j)=\left(\sum _j^{}{T}_{ij}{v}_j\right)}$(T'w'(v))= ∑iwi ∑jTijvj= ∑j( ∑iwiTij)vj$$(T\text{'}w\text{'}(v))=\sum _i^{}{w}_i\sum _j^{}{T}_{ij}{v}_j=\sum _j^{}\left(\sum _i^{}{w}_i{T}_{ij}\right)v_j$$Now we write the linear functional as the product of two matrix: ∑iwiTij${\displaystyle \sum _i^{}{w}_i{T}_{ij}}$. Because this linear is the product of the matrix of dual map and the matrix in the dual space W'${\displaystyle W\text{'}}$, So let's define the matrix of dual map isM(T')=M(T)t$$\mathcal{M}(T\text{'})=\mathcal{M}(T{)}^t$$ Thus ∑iwiTij= ∑iT'jiwi=M(T')w$$\sum _i^{}{w}_i{T}_{ij}=\sum _i^{}T{\text{'}}_{ji}{w}_i=\mathcal{M}(T\text{'})w$$□

The Rank of a Matrix

Definition94row rank & column rank

Suppose A${\displaystyle A}$ is an m-by-n${\displaystyle m-\mathrm{b}\mathrm{y}-n}$ matrix with entries in F${\displaystyle \mathbb{F}}$. The row rank of A${\displaystyle A}$ is the dimensional of the span of the rows of A${\displaystyle A}$ in F1,n${\displaystyle {\mathbb{F}}^{1,n}}$

Theorem95Dimension of range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ equals column rank of M(T)${\displaystyle \mathcal{M}(T)}$

Proof Suppose v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$ and w1,⋯,wm${\displaystyle w_1,\cdots ,w_m}$ is a basis of W${\displaystyle W}$. The function that takes w∈span(Tv1,⋯,Tvn)${\displaystyle w\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(T{v}_1,\cdots ,T{v}_n)}$ to M(w)${\displaystyle \mathcal{M}(w)}$ is easily seen to be an isomorphism from span(Tv1,⋯,Tvn)${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(T{v}_1,\cdots ,T{v}_n)}$ onto span(M(Tv1),⋯,M(Tvn))${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(\mathcal{M}(T{v}_1),\cdots ,\mathcal{M}(T{v}_n))}$, where the last dimension equals the column rank of M(T)${\displaystyle \mathcal{M}(T)}$:a

T11	⋯
T21	⋯
T31	⋯

⋅a

1

0

0

=a

T11

T21

T31

$$\left(\begin{array}{ccc}{T}_{11}& \cdots & \\ T_{21}& \cdots & \\ T_{31}& \cdots & \end{array}\right)\cdot \left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)=\left(\begin{array}{c}{T}_{11}\\ T_{21}\\ T_{31}\end{array}\right)$$It is easy to see that range T=span(Tv1,⋯,Tvn)${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(T{v}_1,\cdots ,T{v}_n)}$. Thus we havedim range T=dim span(Tv1,⋯Tvn)=${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(T{v}_1,\cdots T{v}_n)=}$the column rank of M(T)${\displaystyle \mathcal{M}(T)}$

Theorem96Row rank equals column rank

Suppose A∈Fm,n${\displaystyle A\in {\mathbb{F}}^{m,n}}$. Then the row rank of A${\displaystyle A}$ equals the column rank of A${\displaystyle A}$.Proof Define T: Fn,1→Fm,1${\displaystyle T:{\mathbb{F}}^{n,1}\to {\mathbb{F}}^{m,1}}$ by Tx:=Ax${\displaystyle Tx:=Ax}$. Thus M(T)=A${\displaystyle \mathcal{M}(T)=A}$. Now

	column rank of A	=column rank of M(T)
		=dim range T
		=dim range T'
		=column rank of M(T')
		=column rank of At
		=row rank of A

$$\begin{array}{rl}\mathrm{c}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{m}\mathrm{n}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\mathrm{o}\mathrm{f}A& =\mathrm{c}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{m}\mathrm{n}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\mathrm{o}\mathrm{f}\mathcal{M}(T)\\ & =\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\\ & =\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\text{'}\\ & \mathrm{=}\mathrm{c}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{m}\mathrm{n}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\mathrm{o}\mathrm{f}\mathcal{M}(T\text{'})\\ & =\mathrm{c}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{m}\mathrm{n}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}of{A}^t\\ & \mathrm{=}\mathrm{r}\mathrm{o}\mathrm{w}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{k}\mathrm{o}\mathrm{f}A\end{array}$$

Chapter4. Polynomials(This chapter is mostly omitted)4.A polynomialsThe Division Algorithm for Polynomials

Theorem97Division Algorithm for Polynomials

Suppose that p, s∈P(F)${\displaystyle p,s\in \mathcal{P}(\mathbb{F})}$, with s≠0${\displaystyle s\ne 0}$. Then there exist unique polynomials q,r∈P(F)${\displaystyle q,r\in \mathcal{P}(\mathbb{F})}$ such thatp=sq+r$$p=sq+r$$and deg r<deg s${\displaystyle \mathrm{d}\mathrm{e}\mathrm{g}r<\mathrm{d}\mathrm{e}\mathrm{g}s}$

Theorem98Fundamental Theorem of Algebra

Every non-constant polynomial with complex coefficients has a zero.

⋯${\displaystyle \cdots }$⋯${\displaystyle \cdots }$Chapter5. Eigenvalues, Eigenvectors, and Invariant SubspacesLinear maps from one vector space to another vector space were the objects of study in Chapter 3. Now we begin our investigation of linear maps from a finite-dimensional vector space to itself. Their study constitutes the important part of linear algebra.5.A Invariant SubspacesIn this chapter we develop the tools that will help us understand the structure of operators. Recall that an operator is a linear map from a vector space to itself. Recall also that we denote the set of operators on V${\displaystyle V}$ by L(V)${\displaystyle \mathcal{L}(V)}$. Let's see how we might better understand what an operator looks like. Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. If we have a direct sum decompositionV=U1⊕U2⊕⋯⊕Um,$$V=U_1\oplus U_2\oplus \cdots \oplus U_m,$$where each Uj${\displaystyle U_j}$ is a proper subspace of V${\displaystyle V}$, then to understand the behavior of T${\displaystyle T}$, we need only understand the behavior of each |T||Uj${\displaystyle T{|}_{U_j}}$; here T|Uj${\displaystyle T{|}_{U_j}}$ denotes the restriction of T${\displaystyle T}$ to the smaller domain Uj${\displaystyle U_j}$. Dealing with T|Uj${\displaystyle T{|}_{U_j}}$ should be easier than dealing with T${\displaystyle T}$ because Uj${\displaystyle U_j}$ is a smaller vector space than V${\displaystyle V}$. However, if we intend to apply tools useful in the study of operators (such as taking powers), then we have a problem: T|Uj${\displaystyle T{|}_{U_j}}$ may not map Uj${\displaystyle U_j}$ into itself. Thus we are led to consider only decompositions of V${\displaystyle V}$ of the form above where T${\displaystyle T}$ maps each Uj${\displaystyle U_j}$ into itself.

Definition99invariant subspace

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. A subspace U${\displaystyle U}$ of V${\displaystyle V}$ is called invariant under T${\displaystyle T}$ if u∈U${\displaystyle u\in U}$ implies Tu∈U${\displaystyle Tu\in U}$.

Example99.1 invariant subspace

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Show that each of the following subspaces of V${\displaystyle V}$ is invariant under T${\displaystyle T}$:

★ {0}${\displaystyle \{0\}}$;★ V${\displaystyle V}$;★ null T${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T}$;★ range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$;

Trange Tnull T{0}

Eigenvalues and EigenvectorsNow we turn to an investigation of the simplest possible nontrivial invariant subspaces—invariant subspaces with dimension 1. Take any v∈V${\displaystyle v\in V}$ with V≠0${\displaystyle V\ne 0}$ and let U${\displaystyle U}$ equal the set of all scalar multiples of v${\displaystyle v}$:U={𝜆v:𝜆∈F}=span(v).$$U=\{𝜆v:𝜆\in \mathbb{F}\}=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v).$$Then U${\displaystyle U}$ is a 1-dimensional subspace of V${\displaystyle V}$ and every 1-dimensional subspace of V${\displaystyle V}$ is of this form for an appropriate choice of v${\displaystyle v}$. If U${\displaystyle U}$ is invariant under an operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$, then Tv∈U${\displaystyle Tv\in U}$, and hence there is a scalar 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$ such that Tv=𝜆v.$$Tv=𝜆v.$$ Conversely, if Tv=𝜆v${\displaystyle Tv=𝜆v}$ for some 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$, then span(v)${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v)}$ is a 1-dimensional subspace of V${\displaystyle V}$ invariant under T${\displaystyle T}$. The equationTv=𝜆v$$Tv=𝜆v$$which we have just seen is intimately connected with 1-dimensional invariant subspaces, is important enough that the vectors v${\displaystyle v}$ and scalar 𝜆${\displaystyle 𝜆}$ satisfying it are given special names.

Definition100eigenvalue and eigenvector

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. A number 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$ is called an eigenvalue of T${\displaystyle T}$ if there exist v∈V${\displaystyle v\in V}$ such that v≠0${\displaystyle v\ne 0}$ and Tv=𝜆v${\displaystyle Tv=𝜆v}$. And v${\displaystyle v}$ is called eigenvector, then.

Now we show that eigenvectors corresponding to distinct eigenvalues are linearly independent.

Theorem101Linearly independent eigenvectors

Let T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Suppose 𝜆1,⋯,𝜆m${\displaystyle {𝜆}_1,\cdots ,{𝜆}_m}$ are distinct eigenvalues of T${\displaystyle T}$ and v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ are corresponding eigenvectors. Then v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ is linearly independent.ProofSuppose v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ is linearly dependent. Let k${\displaystyle k}$ be the smallest positive integer such thatvk∈span(v1,⋯,vk-1);$$v_k\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_{k-1});$$then existsvk=a1v1+⋯ak-1vk-1$$v_k=a_1{v}_1+\cdots a_{k-1}{v}_{k-1}$$So the linear map can be written as(8)Tvk=a1𝜆1v1+⋯+ak-1𝜆k-1vk-1$$T{v}_k=a_1{𝜆}_1{v}_1+\cdots +a_{k-1}{𝜆}_{k-1}{v}_{k-1}$$And(9)∵Tvk=𝜆kvk=𝜆k(a1v1+⋯ak-1vk-1), $$\because T{v}_k={𝜆}_k{v}_k={𝜆}_k(a_1{v}_1+\cdots a_{k-1}{v}_{k-1}),$$equ(8) minus equ(9):0=a1(𝜆k-𝜆1)v1+⋯+ak-1(𝜆k-𝜆k-1)vk-1$$0=a_1({𝜆}_k-{𝜆}_1)v_1+\cdots +a_{k-1}({𝜆}_k-{𝜆}_{k-1})v_{k-1}$$

Because we choose k${\displaystyle k}$ to be the smallest positive integer to be linear dependent, thus v1,⋯,vk-1${\displaystyle v_1,\cdots ,v_{k-1}}$ are linear independent. ∴𝜆k=𝜆1=⋯=𝜆k-1$$\therefore {𝜆}_k={𝜆}_1=\cdots ={𝜆}_{k-1}$$Therefore our assumption that v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ is linearly dependent was false. From the sketch we can realize that, unless 𝜆1=𝜆2=𝜆3${\displaystyle {𝜆}_1={𝜆}_2={𝜆}_3}$, there must be𝜆3v3≠Tv3${\displaystyle {𝜆}_3{v}_3\ne T{v}_3}$

Tv3v3𝜆2=6𝜆1=4 Fig4:eigenvectors of different eigenvalues

Theorem102Numbers of eigenvectors

Suppose V${\displaystyle V}$ is finite-dimensional. Then each operator on V${\displaystyle V}$ has at most dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V}$ distinct eigenvalues.Proof Distinct eigenvalues means linear independent vectors.

Restriction and Quotient OperatorsIf T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and U${\displaystyle U}$ is a subspace of V${\displaystyle V}$ invariant under T${\displaystyle T}$, then U${\displaystyle U}$ determines two other operators T|U∈L(U)${\displaystyle T{|}_U\in \mathcal{L}(U)}$ and T/U∈L(V/U)${\displaystyle T/U\in \mathcal{L}(V/U)}$ in a natural way, as defined below

Definition103T|U${\displaystyle T{|}_U}$ and T/U${\displaystyle T/U}$ Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$, and U${\displaystyle U}$ is a subspace of V${\displaystyle V}$ invariant under T${\displaystyle T}$.★ The restriction operator T|U∈L(U)${\displaystyle T{|}_U\in \mathcal{L}(U)}$ is defined byT|U(u):=Tu, u∈U$$T{|}_U(u):=Tu,u\in U$$★ The quotient operator T/U∈L(V/U)${\displaystyle T/U\in \mathcal{L}(V/U)}$ is defined by(T/U)(v+U)=Tv+U$$(T/U)(v+U)=Tv+U$$★ for v∈V${\displaystyle v\in V}$.

(T/U)T

Example103.1

Define an operator T∈L(F2) ${\displaystyle T\in \mathcal{L}\left({\mathbb{F}}^2\right)}$by T(x,y):=(y,0)${\displaystyle T(x,y):=(y,0)}$. Let U={(x,0):x∈F}${\displaystyle U=\{(x,0):x\in \mathbb{F}\}}$. Show that(1) T|U${\displaystyle T{|}_U}$ is the 0${\displaystyle 0}$ operator on U${\displaystyle U}$.(1) solution: obviously(2) there does not exist a subspace W${\displaystyle W}$ of F2${\displaystyle {\mathbb{F}}^2}$ that is invariant under T${\displaystyle T}$ and such that F2=U⊕W${\displaystyle {\mathbb{F}}^2=U\oplus W}$;(2) solution: if there exist such W${\displaystyle W}$, then dim W=dim F2-dim U=1${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}W=\mathrm{d}\mathrm{i}\mathrm{m}{\mathbb{F}}^2-\mathrm{d}\mathrm{i}\mathrm{m}U=1}$. However, it's easy to find the only eigenvector of T${\displaystyle T}$ is 0${\displaystyle 0}$, which is already in U${\displaystyle U}$.(3) T/U${\displaystyle T/U}$ is the 0${\displaystyle 0}$ operator on F2/U${\displaystyle {\mathbb{F}}^2/U}$.(T/U)(v+U)=Tv+U∵Tv∈U∴Tv+U=U, which is the 0 in quotient space F2/U$$\begin{array}{c}(T/U)(v+U)=Tv+U\\ \because Tv\in U\\ \therefore Tv+U=U,whichisthe0inquotientspace{\mathbb{F}}^2/U\end{array}$$

5.B Eigenvectors and Upper-Triangular MatricesPolynomials Applied to OperatorsThis is a new use of the symbol p${\displaystyle p}$ because we are applying it to operators, not just elements of F${\displaystyle \mathbb{F}}$.

Definition104p(T)${\displaystyle p(T)}$

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$, and p∈P(F)${\displaystyle p\in \mathcal{P}(\mathbb{F})}$ is a polynomial given byp(z)=a0+a1z+⋯+amzm$$p(z)=a_0+a_{1}z+\cdots +a_m{z}^m$$for z∈F${\displaystyle z\in \mathbb{F}}$. Then p(T)${\displaystyle p(T)}$ is the operator defined byp(T)=a0I+a1T+a2T2+⋯+amTm.$$p(T)=a_{0}I+a_{1}T+a_2{T}^2+\cdots +a_m{T}^m.$$

If we fix an operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ , then the function from P(F)${\displaystyle \mathcal{P}(\mathbb{F})}$ to L(V)${\displaystyle \mathcal{L}(V)}$ given by p↦p(T)${\displaystyle p\mapsto p(T)}$ is linear.

Definition105product of polynomials

If p,q∈P(F)${\displaystyle p,q\in \mathcal{P}(\mathbb{F})}$, then p q∈P(F)${\displaystyle pq\in \mathcal{P}(\mathbb{F})}$ is the polynomial defined by(pq)(z)=p(z)q(z)$$(pq)(z)=p(z)q(z)$$for z∈F${\displaystyle z\in \mathbb{F}}$. And any tow polynomials of an operator commute:(pq)T=p(T)q(T)=q(T)p(T)$$(pq)T=p(T)q(T)=q(T)p(T)$$

Existence of Eigenvalues

Theorem106Operators on complex vector spaces have an eigenvalue

Every operator on a finite-dimensional, nonzero, complex vector space has an eigenvalue.Proof Suppose V${\displaystyle V}$ is a complex vector space with dimension n${\displaystyle n}$ and operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Choose v∈V ${\displaystyle v\in V}$with v≠${\displaystyle v\ne }$0, Thenv,Tv,T2v,⋯Tnv$$v,Tv,T^{2}v,\cdots T^{n}v$$is not linear independent, because they are more than n${\displaystyle n}$. Thus there exist complex number a0,⋯,an${\displaystyle a_0,\cdots ,a_n}$, not all 0, such that0=a0v+a1Tv+⋯anTnv$$0=a_{0}v+a_{1}Tv+\cdots a_n{T}^{n}v$$Make the a's${\displaystyle a\text{'}s}$ the coefficients of a polynomial, which by the Fundamental Theorem of Algebra has a factorizationa0+a1z+⋯+anzn=c(z-𝜆1)⋯(z-𝜆m),$$a_0+a_{1}z+\cdots +a_n{z}^n=c(z-{𝜆}_1)\cdots (z-{𝜆}_m),$$where c ${\displaystyle c}$is a nonzero complex number, each 𝜆j${\displaystyle {𝜆}_j}$ is in C${\displaystyle \mathbb{C}}$, and the equation holds for all z∈C${\displaystyle z\in \mathbb{C}}$. We then havea

0	=a0v+⋯anTnv
	=(a0I+a1T+⋯+anTn)v
	=c(T-𝜆1I)⋯(T-𝜆mI)v.

$$\begin{array}{rl}0& =a_{0}v+\cdots a_n{T}^{n}v\\ & =(a_{0}I+a_{1}T+\cdots +a_n{T}^n)v\\ & =c(T-{𝜆}_1\mathbb{I})\cdots (T-{𝜆}_m\mathbb{I})v.\end{array}$$

Upper-Triangular MatricesNow that we are studying operators, which map a vector space to itself, the emphasis is on using only one basis.

Definition107Matrix of Operator

Tvk=A1,kv1+⋯+An,kvn${\displaystyle T{v}_k=A_{1,k}{v}_1+\cdots +A_{n,k}{v}_n}$

Definition108Diagonal of a Matrix

The diagonal of a matrix consists of the entries along the line from the upper left corner to the bottom right corner.

Definition109Upper-triangular matrix

If all the entries below the diagonal equal 0.

Theorem110Conditions for upper-triangular matrix

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and v1,⋯vm${\displaystyle v_1,\cdots v_m}$ is a basis of V${\displaystyle V}$. Then the following are equivalent:★ the matrix of T${\displaystyle T}$ with respect to v1,⋯vn${\displaystyle v_1,\cdots v_n}$ is upper triangular;★ Tvj∈span(v1,⋯,vj)${\displaystyle T{v}_j\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_j)}$ for each j=1,⋯,n${\displaystyle j=1,\cdots ,n}$;★ span(v1,⋯,vj)${\displaystyle \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_j)}$ is invariant under T${\displaystyle T}$ for each j=1,⋯,n${\displaystyle j=1,\cdots ,n}$.

Theorem111Over C${\displaystyle \mathbb{C}}$, every operator has an upper-triangular matrix

Suppose V${\displaystyle V}$ is a finite-dimensional complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then T${\displaystyle T}$ has an upper-triangular matrix with respect to some basis of V${\displaystyle V}$.Proof Use induction on the dimension of V${\displaystyle V}$. Clearly the desired result holds if dim V=1${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V=1}$.Suppose now that dimV>1${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V>1}$ and the desired results holds for all complex vector spaces whose dimension is less than the dimension of V.${\displaystyle V.}$ Let 𝜆${\displaystyle 𝜆}$ be any eigenvalue of T${\displaystyle T}$. LetU=range(T-𝜆I)$$U=\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}(T-𝜆\mathbb{I})$$

Because T-𝜆I${\displaystyle T-𝜆\mathbb{I}}$ is not surjective, thus dim U<dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}U<\mathrm{d}\mathrm{i}\mathrm{m}V}$. Furthermore, U${\displaystyle U}$ is invariant under T${\displaystyle T}$. To prove this, suppose u∈U.${\displaystyle u\in U.}$ Then a Tu =(T-𝜆I)u⏠⏣⏣⏣⏡⏣⏣⏣⏢∈U+𝜆u⏠⏣⏣⏡⏣⏣⏢∈U∈U $$\begin{array}{rl}Tu& =\underset{\in U}{\underbrace{(T-𝜆\mathbb{I})u}}+\underset{\in U}{\underbrace{𝜆u}}\in U\end{array}$$Thus T|U${\displaystyle T{|}_U}$ is an operator on U${\displaystyle U}$. By our induction hypothesis, there is a basis uj${\displaystyle u_j}$ based which T|U${\displaystyle T{|}_U}$ has upper-triangular matrix.

injective and surjective is equivalent for L(V)${\displaystyle \mathcal{L}(V)}$ in finite-dimensional space, and eigenvalue means there is at least a 1-dimensional invariant subspace.

Tuj=(T|U)(uj)∈span(u1,⋯,uj)$$T{u}_j=(T{|}_U)(u_j)\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(u_1,\cdots ,u_j)$$Extend u1,⋯,um${\displaystyle u_1,\cdots ,u_m}$ to a basis u1,⋯,um, v1,⋯,vn${\displaystyle u_1,\cdots ,u_m,v_1,\cdots ,v_n}$ of V${\displaystyle V}$. For each k${\displaystyle k}$, we have a obvious relation:Tvk≡(T-𝜆I)vk+𝜆vk$$T{v}_k\equiv (T-𝜆\mathbb{I})v_k+𝜆v_k$$The definition of U${\displaystyle U}$ shows that (T-𝜆I)vk∈U=span(u1,⋯,um)${\displaystyle (T-𝜆\mathbb{I})v_k\in U=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(u_1,\cdots ,u_m)}$. Thus the equation above shows that Tvk∈span(u1,⋯,um,vk)$$T{v}_k\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(u_1,\cdots ,u_m,v_k)$$So, it definitely also has:Tvk∈span(u1,⋯,um,v1,⋯,vk)$$T{v}_k\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(u_1,\cdots ,u_m,v_1,\cdots ,v_k)$$

To show the proof above more clearly, I drew some sketches.

The key idea is to use subspace with a smaller dimension, and the space we are interesting is V${\displaystyle V}$.Vrange T(V)domainTHowever, we actually only need to prove the vector in range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ can be represented by some basis. (Pretend range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ has a smaller dimension)range T(range T)If we investigate range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$, and treat it as the domain, we can use induction. We may loose a part of rangeT(V)${\displaystyle T(V)}$ . And that's the part we need to prove.VT'=T-𝜆Irange T'(V)domainTHow can we make sure range T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ has a smaller dimension? We know that with a eigenvalue, there must be a T'=T-𝜆I${\displaystyle T\text{'}=T-𝜆\mathbb{I}}$, such that it is not injective. Use it and redo the proof we did to T${\displaystyle T}$.T

Theorem112Determination of invertibility from upper-triangular matrix

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ has an upper-triangular matrix with respect to some basis of V${\displaystyle V}$. Then T${\displaystyle T}$ is invertible if and only if all the entries on the diagonal of that upper-triangular matrix are nonzero.ProofTo prove invertibility, we need to prove injectivity or surjectivity. Suppose the upper-triangular matrix has 𝜆1,⋯,𝜆n${\displaystyle {𝜆}_1,\cdots ,{𝜆}_n}$ on its diagonal, with basis v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$.If 𝜆i≠0${\displaystyle {𝜆}_i\ne 0}$:then we can have T(v1

𝜆1)=v1${\displaystyle T\left(\frac{v_1}{{𝜆}_1}\right)=v_1}$, T(v2

𝜆2)=av1+v2${\displaystyle T\left(\frac{v_2}{{𝜆}_2}\right)=a{v}_1+v_2}$, ⋯${\displaystyle \cdots }$Because 𝜆≠0${\displaystyle 𝜆\ne 0}$, we can construct such n${\displaystyle n}$ equations, each equation maps a vector into linearly independent vector list. So range T=dim V${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T=\mathrm{d}\mathrm{i}\mathrm{m}V}$. To prove the other direction, now suppose that T${\displaystyle T}$ is invertible. This implies that 𝜆1≠0${\displaystyle {𝜆}_1\ne 0}$, because otherwise we would have Tv1=0${\displaystyle T{v}_1=0}$. Suppose 𝜆j=0,1<j⩽n${\displaystyle {𝜆}_j=0,1<j⩽n}$, if 𝜆j=0${\displaystyle {𝜆}_j=0}$, it means Tvj∈span(v1,⋯,vj-1)$$T{v}_j\in \mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_{j-1})$$So list {Tv1,Tv2,⋯,Tvn}${\displaystyle \{T{v}_1,T{v}_2,\cdots ,T{v}_n\}}$ cannot be linear independent. T${\displaystyle T}$ is not surjective and are not invertible.

Now we have a way to compute the eigenvalues from a upper-triangular matrix:

Theorem113Determination of eigenvalues from upper-triangular matrix

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and has an upper-triangular matrix with respect to some basis of V${\displaystyle V}$. Then the eigenvalues of T${\displaystyle T}$ are precisely the entries on the diagonal of the upper-triangular matrix.Proof Use the invertibility of T-𝜆I${\displaystyle T-𝜆\mathbb{I}}$ for the upper-triangular matrix.

5.C Eigenspaces and Diagonal Matrices

Definition114diagonal matrix

If all the entries except the diagonal equal 0.

Definition115eigenspace, E(𝜆,T)${\displaystyle \mathbb{E}(𝜆,T)}$

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$. The eigenspace of T${\displaystyle T}$ corresponding to 𝜆${\displaystyle 𝜆}$, denoted E(𝜆,T)${\displaystyle \mathbb{E}(𝜆,T)}$, is defined byE(𝜆,T)=null(T-𝜆I)$$\mathbb{E}(𝜆,T)=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(T-𝜆\mathbb{I})$$In other words, E(𝜆,T)${\displaystyle \mathbb{E}(𝜆,T)}$ is the set of all eigenvectors of T${\displaystyle T}$ corresponding to eigenvalue 𝜆${\displaystyle 𝜆}$, along with the 0${\displaystyle 0}$ vector.

Theorem116Sum of eigenspaces is a direct sum

Suppose V${\displaystyle V}$ is finite-dimensional and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Suppose also that 𝜆1,⋯,𝜆m${\displaystyle {𝜆}_1,\cdots ,{𝜆}_m}$ are distinct eigenvalues of T${\displaystyle T}$. ThenE(𝜆1,T)+⋯+E(𝜆m,T)$$\mathbb{E}({𝜆}_1,T)+\cdots +\mathbb{E}({𝜆}_m,T)$$is a direct sum. Furthermore,dim E(𝜆1,T)+⋯+dim E(𝜆,T)⩽dim V$$\mathrm{d}\mathrm{i}\mathrm{m}\mathbb{E}({𝜆}_1,T)+\cdots +\mathrm{d}\mathrm{i}\mathrm{m}\mathbb{E}(𝜆,T)⩽\mathrm{d}\mathrm{i}\mathrm{m}V$$

Definition117diagonalizable

An operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is called diagonalizable if the operator has a diagonal matrix with respect to some basis of V${\displaystyle V}$.

Theorem118Conditions equivalent to diagonalizability

Suppose V${\displaystyle V}$ is finite-dimensional and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Suppose also that 𝜆1,⋯,𝜆m${\displaystyle {𝜆}_1,\cdots ,{𝜆}_m}$ are distinct eigenvalues of T${\displaystyle T}$. Then the following are equivalent:(1) T${\displaystyle T}$ is diagonalizable;(2) V${\displaystyle V}$ has a basis consisting of eigenvectors of T${\displaystyle T}$;(3) there exist 1-dimensional subspaces U1,⋯,Un${\displaystyle U_1,\cdots ,U_n}$ of V${\displaystyle V}$, each invariant under T${\displaystyle T}$, such that(10)V=U1⊕⋯⊕Un;$$V=U_1\oplus \cdots \oplus U_n;$$(4) (11)V=E(𝜆1,T)⊕⋯⊕E(𝜆m,T)$$V=\mathbb{E}({𝜆}_1,T)\oplus \cdots \oplus \mathbb{E}({𝜆}_m,T)$$(5) (12)dim V=dim E(𝜆1,T)+⋯+dim E(𝜆m,T)$$\mathrm{d}\mathrm{i}\mathrm{m}V=\mathrm{d}\mathrm{i}\mathrm{m}\mathbb{E}({𝜆}_1,T)+\cdots +\mathrm{d}\mathrm{i}\mathrm{m}\mathbb{E}({𝜆}_m,T)$$Proof An operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ has a diagonal matrix with respect to a basis v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ of V${\displaystyle V}$ if and only if Tvj${\displaystyle T{v}_j}$ =𝜆jvj${\displaystyle ={𝜆}_j{v}_j}$ for each j${\displaystyle j}$. Thus (1) and (2) are equivalent.Suppose (2) holds; thus V${\displaystyle V}$ has a basis v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ consisting of eigenvectors of T${\displaystyle T}$. For each j${\displaystyle j}$, letUj=span(vj)$$U_j=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_j)$$Thus V=U1⊕⋯⊕Un${\displaystyle V=U_1\oplus \cdots \oplus U_n}$, (3) holds.Suppose now that (3) holds. The Uj${\displaystyle U_j}$ respected to the same 𝜆${\displaystyle 𝜆}$ can consist E(𝜆m,T)${\displaystyle \mathbb{E}({𝜆}_m,T)}$. (4) holds, eigenvectors belong different eigenvalues are linearly independent. So (12) also holds.

Now we have shown that (1) (2) (3) are equivalent, and (3) implies (4), implies (5), implies (6). We will show that (5) implies (2). Suppose (5) holds, which means equ(12) holds. Choose a basis of each E(𝜆j,T)${\displaystyle \mathbb{E}({𝜆}_j,T)}$; put all these bases together to form a list v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ of eigenvectors of T${\displaystyle T}$, where n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. And we know that they are linearly independent, and are all eigenvectors of T${\displaystyle T}$. Thus (2) holds.

Tv3v3𝜆2=6𝜆1=4

Theorem119Enough eigenvalues implies diagonalizability

If T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ has dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V}$ distinct eigenvalues, then T${\displaystyle T}$ is diagonalizable.

Chapter6. Inner Product SpacesIn making the definition of a vector space, we generalized the linear structure (addition and scalar multiplication) of R2${\displaystyle {\mathbb{R}}^2}$ and R3${\displaystyle {\mathbb{R}}^3}$. We ignored other important features, such as the notions of length and angle. These ideas are embedded in the concept we now investigate, inner products. 6.A Inner Products and NormsInner Products

Definition120dot product

For x,y∈Rn${\displaystyle x,y\in {\mathbb{R}}^n}$, the dot product of x${\displaystyle x}$ and y${\displaystyle y}$, denoted x⋅y${\displaystyle x\cdot y}$, is defined byx⋅y=x1y1+⋯+xnyn,$$x\cdot y=x_1{y}_1+\cdots +x_n{y}_n,$$where x=(x1,⋯,xn)${\displaystyle x=(x_1,\cdots ,x_n)}$ and y=(y1,⋯,yn)${\displaystyle y=(y_1,\cdots ,y_n)}$.

Note that the dot product of two vectors in Rn${\displaystyle {\mathbb{R}}^n}$ is a number, not a vector. Obviously x⋅x=‖x‖2${\displaystyle x\cdot x=\Vert x{\Vert }^2}$ for all x∈Rn${\displaystyle x\in {\mathbb{R}}^n}$. The dot product on Rn${\displaystyle {\mathbb{R}}^n}$ has the following properties:★ x⋅x⩾0${\displaystyle x\cdot x⩾0}$ ★ x⋅x=0${\displaystyle x\cdot x=0}$ if and only if x=0${\displaystyle x=0}$;★ for y∈Rn${\displaystyle y\in {\mathbb{R}}^n}$ fixed, the map from Rn${\displaystyle {\mathbb{R}}^n}$ to R${\displaystyle \mathbb{R}}$ that sends x∈Rn${\displaystyle x\in {\mathbb{R}}^n}$ to x⋅y${\displaystyle x\cdot y}$ is linear;★ x⋅y=y⋅x${\displaystyle x\cdot y=y\cdot x}$ for all x,y∈Rn${\displaystyle x,y\in {\mathbb{R}}^n}$ An inner product is a generalization of the dot product. At this point you may be tempted to guess that an inner product is defined by abstracting the properties of the dot product discussed in the last paragraph. For real vector spaces, that guess is correct. However, so that we can make a definition that will be useful for both real and complex vector spaces, we need to examine the complex case before making the definition.‖z‖=ℜz2+ℑz2=z⏨z$$\Vert z\Vert =\sqrt{\Re z^2+\Im z^2}=z\overline{z}$$

Definition121inner product

An inner product on V${\displaystyle V}$ is a function that takes each ordered pair (u,v)${\displaystyle (u,v)}$ of elements of V${\displaystyle V}$ to a number ⟨u,v>∈F${\displaystyle ⟨u,v⟩\in \mathbb{F}}$ and has the following properties:positivity⟨v,v>⩾0 for all v∈V;$$⟨v,v⟩⩾0\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}v\in V;$$definiteness⟨v,v>=0 if and only if v=0;$$⟨v,v⟩=0\mathrm{i}\mathrm{f}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{o}\mathrm{n}\mathrm{l}\mathrm{y}\mathrm{i}\mathrm{f}v=0;$$additivity in first slot⟨u+v,w>=⟨u,w>+⟨v,w> for all u,v,w∈V$$⟨u+v,w⟩=⟨u,w⟩+⟨v,w⟩\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}u,v,w\in V$$homogeneity in first slot⟨𝜆u,v>=𝜆⟨u,v> for all 𝜆∈F and all u,v∈V;$$⟨𝜆u,v⟩=𝜆⟨u,v⟩\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{𝜆}\mathrm{\in }\mathbb{F}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{l}\mathrm{l}u,v\in V;$$conjugate symmetry⟨u,v>=⏨⏨⏨⟨v,u> for all u,v∈V.$$⟨u,v⟩=\overline{⟨v,u⟩}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}u,v\in V.$$

Example121.1

An inner product can be defined on P(R)${\displaystyle \mathcal{P}(\mathbb{R})}$ by⟨p,q>=∞∫0p(x)q(x)ⅇ-x dx.$$⟨p,q⟩=\underset{0}{\overset{\infty }{\int }}p(x)q(x)e^{-x}\mathrm{d}x.$$

Definition122inner product space

An inner product space is a vector space V${\displaystyle V}$ along with an inner product on V${\displaystyle V}$.

For the rest of this chapter, V${\displaystyle V}$ denotes an inner product space over F${\displaystyle \mathbb{F}}$.

Theorem123basic properties of an inner product

★ For each fixed u∈V${\displaystyle u\in V}$, the function that takes v${\displaystyle v}$ to ⟨v,u>${\displaystyle ⟨v,u⟩}$is a linear map from V${\displaystyle V}$ to F${\displaystyle \mathbb{F}}$.★ But fix v${\displaystyle v}$ is not a linear map!★ ⟨0,u>=⟨u,0>=0${\displaystyle ⟨0,u⟩=⟨u,0⟩=0}$★ ⟨u,v+w>=⟨u,v>+⟨u,w>${\displaystyle ⟨u,v+w⟩=⟨u,v⟩+⟨u,w⟩}$★ ⟨u,𝜆v>=⏨𝜆⟨u,v>${\displaystyle ⟨u,𝜆v⟩=\overline{𝜆}⟨u,v⟩}$

NormsNow we see that each inner product determines a norm

Definition124norm, ‖v‖${\displaystyle \Vert v\Vert }$

For v∈V${\displaystyle v\in V}$, the norm of v${\displaystyle v}$, is defined by‖v‖=⟨v,v>.$$\Vert v\Vert =\sqrt{⟨v,v⟩}.$$

Theorem125basic properties of the norm

Suppose v∈V${\displaystyle v\in V}$.(1) ‖v‖=0${\displaystyle \Vert v\Vert =0}$ if and only if v=0${\displaystyle v=0}$(2) ‖𝜆v‖=||𝜆||‖v‖${\displaystyle \Vert 𝜆v\Vert =|𝜆|\Vert v\Vert }$ for all 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$

Definition126orthogonal

Two vectors u,v∈V${\displaystyle u,v\in V}$ are called orthogonal if ⟨u,v>=0${\displaystyle ⟨u,v⟩=0}$.

You can think of the word orthogonal as a fancy word meaning perpendicular. We begin our study of orthogonal with an easy result.

Theorem127Orthogonal and 0${\displaystyle 0}$

★ 0${\displaystyle 0}$ is orthogonal to every vector in V${\displaystyle V}$.★ 0${\displaystyle 0}$ is the only vector in V${\displaystyle V}$ that is orthogonal to itself.

Theorem128Pythagorean Theorem

Suppose v,u${\displaystyle v,u}$ are orthogonal vectors in V${\displaystyle V}$. Then‖u+v‖2=‖u‖2+‖v‖2.$$\Vert u+v{\Vert }^2=\Vert u{\Vert }^2+\Vert v{\Vert }^2.$$Proof a

‖u+v‖2	=⟨u+v,u+v>
	=⟨u,v>+⟨u,u>+⟨v,u>+⟨v,v>

$$\begin{array}{rl}\Vert u+v{\Vert }^2& =⟨u+v,u+v⟩\\ & =⟨u,v⟩+⟨u,u⟩+⟨v,u⟩+⟨v,v⟩\end{array}$$□

Theorem129An orthogonal decomposition

Suppose v,u∈V${\displaystyle v,u\in V}$, with v≠0${\displaystyle v\ne 0}$. Set c=⟨u,v>

‖v‖2${\displaystyle c=\frac{⟨u,v⟩}{\Vert v{\Vert }^2}}$ and w=u-⟨u,v>

‖v‖2v${\displaystyle w=u-\frac{⟨u,v⟩}{\Vert v{\Vert }^2}v}$. Then⟨w,v>=0 and u=cv+w.$$⟨w,v⟩=0\mathrm{a}\mathrm{n}\mathrm{d}u=cv+w.$$

Theorem130Cauchy-Schwarz Inequality

Suppose v,u∈V${\displaystyle v,u\in V}$. Then||⟨u,v>||⩽‖u‖ ‖v‖.$$|⟨u,v⟩|⩽\Vert u\Vert \Vert v\Vert .$$Proof u=w+⟨u,v>

‖v‖2v$$u=w+\frac{⟨u,v⟩}{\Vert v{\Vert }^2}v$$because w${\displaystyle w}$ and v${\displaystyle v}$ are orthogonal, so the norm of u${\displaystyle u}$ can be written asa

‖u‖	=‖⟨u,v> ‖v‖2v‖2+‖w‖2
	=||⟨u,v>||2 ‖v‖2+‖w‖2
	⩾||⟨u,v>||2 ‖v‖2

$$\begin{array}{rl}\Vert u\Vert & =\Vert \frac{⟨u,v⟩}{\Vert v{\Vert }^2}v{\Vert }^2+\Vert w{\Vert }^2\\ & =\frac{|⟨u,v⟩{|}^2}{\Vert v{\Vert }^2}+\Vert w{\Vert }^2\\ & ⩾\frac{|⟨u,v⟩{|}^2}{\Vert v{\Vert }^2}\end{array}$$

Example130.1examples of the Cauchy-Schwarz Inequality

(1) ||x1y1+⋯+xnyn||2⩽(x21+⋯+x2n)(y21+⋯+y2n)$$|x_1{y}_1+\cdots +x_n{y}_n{|}^2⩽(x_1^2+\cdots +x_n^2\left)\right(y_1^2+\cdots +y_n^2)$$(2) |1∫-1f(x)g(x)dx|2⩽(1∫-1(f(x))2dx)(1∫-1(g(x))2dx).$$|\underset{-1}{\overset{1}{\int }}f(x)g(x)\mathrm{d}x{|}^2⩽(\underset{-1}{\overset{1}{\int }}(f(x){)}^2\mathrm{d}x)\left(\underset{-1}{\overset{1}{\int }}\right(g(x){)}^2\mathrm{d}x).$$

Theorem131Triangular Inequality

Suppose v,u∈V${\displaystyle v,u\in V}$. Then‖u+v‖⩽‖u‖+‖v‖.$$\Vert u+v\Vert ⩽\Vert u\Vert +\Vert v\Vert .$$Proof a

‖u+v‖2	=⟨u+v,u+v>
	=⟨u,u>+⟨v,v>+⟨u,v>+⟨v,u>
	=⟨u,u>+⟨v,v>+⟨u,v>+⏨⏨⏨⟨u,v>
	=⟨u,u>+⟨v,v>+2ℜ⟨u,v>
	⩽‖u‖2+‖v‖2+2||⟨u,v>||
	⩽‖u‖2+‖v‖2+2‖u‖ ‖v‖
	=(‖u‖+‖v‖)2

$$\begin{array}{rl}\Vert u+v{\Vert }^2& =⟨u+v,u+v⟩\\ & =⟨u,u⟩+⟨v,v⟩+⟨u,v⟩+⟨v,u⟩\\ & =⟨u,u⟩+⟨v,v⟩+⟨u,v⟩+\overline{⟨u,v⟩}\\ & =⟨u,u⟩+⟨v,v⟩+2\Re ⟨u,v⟩\\ & ⩽\Vert u{\Vert }^2+\Vert v{\Vert }^2+2|⟨u,v⟩|\\ & ⩽\Vert u{\Vert }^2+\Vert v{\Vert }^2+2\Vert u\Vert \Vert v\Vert \\ & =(\Vert u\Vert +\Vert v\Vert {)}^2\end{array}$$

The next result is called the parallelogram equality because of its geometric interpretation: in every parallelogram, the sum of the squares of the lengths of the diagonals equals the sum of the squares of the lengths of the four sides.

vuuvu-vu+v

Theorem132Parallelogram Inequality

Suppose v,u∈V${\displaystyle v,u\in V}$. Then‖u+v‖2+‖u-v‖2=2(‖u‖2+‖v‖2).$$\Vert u+v{\Vert }^2+\Vert u-v{\Vert }^2=2(\Vert u{\Vert }^2+\Vert v{\Vert }^2).$$ProofWe havea

‖u+v‖2-‖u-v‖2	=⟨u+v,u+v>+⟨u-v,u-v>
	=‖u‖2+‖v‖2+||⟨u,v>||+⟨v,u>+‖u‖2+‖v‖2-||⟨u,v>||-||⟨v,u>||
	=2(‖u‖2+‖v‖2)2

$$\begin{array}{rl}\Vert u+v{\Vert }^2-\Vert u-v{\Vert }^2& =⟨u+v,u+v⟩+⟨u-v,u-v⟩\\ & =\Vert u{\Vert }^2+\Vert v{\Vert }^2+|⟨u,v⟩|+⟨v,u⟩+\Vert u{\Vert }^2+\Vert v{\Vert }^2-|⟨u,v⟩|-|⟨v,u⟩|\\ & =2(\Vert u{\Vert }^2+\Vert v{\Vert }^2{)}^2\end{array}$$

6.B Orthonormal Bases

Definition133Orthonormal

A list of vectors is called orthonormal if each vector in the list has norm 1 and is orthogonal to all the other vectors in the list. In other words, a list e1,⋯,em${\displaystyle e_1,\cdots ,e_m}$ of vectors in V${\displaystyle V}$ is orthonormal if⟨ej,ek>=a

1		if j=k
0		if j≠k

$$⟨e_j,e_k⟩=\{\begin{array}{ll}1& \mathrm{i}\mathrm{f}j=k\\ 0& \mathrm{i}\mathrm{f}j\ne k\end{array}$$

Theorem134An orthonormal list is linearly independent

Every orthonormal list of vectors is linearly independent.Proofmake the inner product between a vector and other vectors in the list

How do we go and finding orthonormal bases? The algorithm used in the next proof is called the Gram-Schmidt Procedure. It gives a method for turning a linearly independent list into an orthonormal list with the same span as the origin list.

Theorem135Gram-Schmidt Procedure

Suppose v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ is linearly independent. Let e1=v1/‖v1‖${\displaystyle e_1=v_1/\Vert v_1\Vert }$. For j=2,⋯,m${\displaystyle j=2,\cdots ,m}$, define ej${\displaystyle e_j}$ inductively byej=vj-j-1∑i=1⟨vj,ei>ei

‖vj-j-1∑i=1⟨vj,ei>ei‖$$e_j=\frac{v_j-\sum _{i=1}^{j-1}⟨v_j,e_i⟩e_i}{\Vert v_j-\sum _{i=1}^{j-1}⟨v_j,e_i⟩e_i\Vert }$$Then e1,⋯,em${\displaystyle e_1,\cdots ,e_m}$ is an orthonormal list of vectors in V${\displaystyle V}$ such thatspan(v1,⋯,vj)=span(e1,⋯,ej)$$\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(v_1,\cdots ,v_j)=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(e_1,\cdots ,e_j)$$for j=1,⋯,m.${\displaystyle j=1,\cdots ,m.}$ProofIt's easy to verify ⟨ej,ei>=0${\displaystyle ⟨e_j,e_i⟩=0}$. To think this intuitively:It is removing the components projected into the previous vector list.

Theorem136Existence of orthonormal basis

Every finite-dimensional inner product space has an orthonormal basis. Just apply the Gram-Schmidt Procedure to a basis of the space.

Theorem137Orthonormal lists extend to orthonormal basis

Suppose V${\displaystyle V}$ is finite-dimensional. Then every orthonormal list of vectors in V${\displaystyle V}$ can be extended to an orthonormal basis of V${\displaystyle V}$.

a

𝜆1	x	z
	𝜆2	y
		𝜆3

a

v1

v2

v3

a

	e1=v1/𝜆2
	e2=av2+bv1
	e3=cv1+bv2+dv3

Schmidt Procedure only involves basis before the perticular vector

Theorem138Upper-triangular matrix with respect to orthonormal basis

T${\displaystyle T}$ has an upper-triangular matrix with respect to some orthonormal basis of V${\displaystyle V}$ as long as it has an upper-triangular matrix with respect to some basis.

The next result is an important application of the result above

Theorem139Schur's Theorem

Suppose V${\displaystyle V}$ is a finite-dimensional complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then T${\displaystyle T}$ has an upper-triangular matrix with respect to some orthonormal basis of V${\displaystyle V}$.

Linear Functional on Inner Product SpacesRecall our definition of linear functional. If u∈V${\displaystyle u\in V}$, then the map that sends v${\displaystyle v}$ to ⟨v,u>${\displaystyle ⟨v,u⟩}$ is a linear functional on V${\displaystyle V}$. The next result shows that every linear functional on V${\displaystyle V}$ is of this form.

Theorem140Riesz Representation Theorem

Suppose V${\displaystyle V}$ is finite-dimensional and 𝜑${\displaystyle 𝜑}$ is a linear functional on V${\displaystyle V}$. Then there is a unique vector u∈V${\displaystyle u\in V}$ such that𝜑(v)=⟨v,u>$$𝜑(v)=⟨v,u⟩$$for every v∈V${\displaystyle v\in V}$Proof First we show there exists a vector u∈V${\displaystyle u\in V}$ such that 𝜑(v)=⟨v,u>${\displaystyle 𝜑(v)=⟨v,u⟩}$. a

𝜑(v)	=𝜑 ∑i⟨v,ei>ei
	=⟨v,e1>𝜑(e1)+⋯+⟨v,en>𝜑(en)
	=⟨v, ⏨⏨⏨𝜑(e1)e1>+⋯+⟨v,⏨⏨⏨𝜑(en)en>
	=⟨v, ⏨⏨⏨𝜑(e1)e1+⋯+⏨⏨⏨𝜑(en)en>
	=⟨v,u>
u	=⏨⏨⏨𝜑(e1)e1+⋯⏨⏨⏨𝜑(en)en

$$\begin{array}{rl}𝜑(v)& =𝜑\sum _i^{}⟨v,e_i⟩e_i\\ & =⟨v,e_1⟩𝜑(e_1)+\cdots +⟨v,e_n⟩𝜑(e_n)\\ & =⟨v,\overline{𝜑(e_1)}{e}_1⟩+\cdots +⟨v,\overline{𝜑(e_n)}{e}_n⟩\\ & =⟨v,\overline{𝜑(e_1)}{e}_1+\cdots +\overline{𝜑(e_n)}{e}_n⟩\\ & =⟨v,u⟩\\ u& =\overline{𝜑(e_1)}{e}_1+\cdots \overline{𝜑(e_n)}{e}_n\end{array}$$Then we show u${\displaystyle u}$ is unique:(13)if 𝜑(v)=⟨v,u1>=⟨v,u2>$$\mathrm{i}\mathrm{f}𝜑(v)=⟨v,u_1⟩=⟨v,u_2⟩$$(13)-(13):0=⟨v,u1-u2>$$0=⟨v,u_1-u_2⟩$$Because (13) should holds for all v∈V${\displaystyle v\in V}$, then u1-u2${\displaystyle u_1-u_2}$ is orthogonal with all v${\displaystyle v}$. It must be 0${\displaystyle 0}$. Thus u1${\displaystyle u_1}$ equals u2${\displaystyle u_2}$.

6.C Orthogonal Complements and Minimization ProblemsOrthogonal Complements

Definition141orthogonal complement, U⊥${\displaystyle U^{\perp }}$

If U${\displaystyle U}$ is a subset of V${\displaystyle V}$, then the orthogonal complement of U${\displaystyle U}$, denoted U⊥${\displaystyle U^{\perp }}$, is the set of all vectors in V${\displaystyle V}$ that are orthogonal to every vector in U${\displaystyle U}$:U⊥={v∈V:⟨v,u>=0 for every u∈U}.$$U^{\perp }=\{v\in V:⟨v,u⟩=0\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{e}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{y}u\in U\}.$$

Theorem142Basic properties of orthogonal complement

(1) If U${\displaystyle U}$ is a subset of V${\displaystyle V}$, then U⊥${\displaystyle U^{\perp }}$ is a subspace of V${\displaystyle V}$.(2) {0}⊥=V${\displaystyle \{0{\}}^{\perp }=V}$.(3) V⊥={0}${\displaystyle V^{\perp }=\{0\}}$.(4) If U${\displaystyle U}$ is a subset of V${\displaystyle V}$, then U∩U⊥⊂{0}${\displaystyle U\cap U^{\perp }\subset \{0\}}$.(5) If U${\displaystyle U}$ and W${\displaystyle W}$ are subsets of V${\displaystyle V}$ and U⊂W${\displaystyle U\subset W}$, then W⊥⊂U⊥${\displaystyle W^{\perp }\subset U^{\perp }}$.

Theorem143Direct sum of a subspace and its orthogonal complement

Suppose U${\displaystyle U}$ is a finite-dimensional subspace of V${\displaystyle V}$. Then(14)V=U⊕U⊥.$$V=U\oplus U^{\perp }.$$ProofLet e1,⋯,em${\displaystyle e_1,\cdots ,e_m}$ be an orthonormal basis of U${\displaystyle U}$. for a vector in V${\displaystyle V}$, we can decompose it into two parts:v=⟨v,e1>e1+⋯+⟨v,em>em⏠⏣⏣⏣⏣⏣⏣⏡⏣⏣⏣⏣⏣⏣⏢u+v-⟨v,e1>e1+⋯+⟨v,em>em⏠⏣⏣⏣⏣⏣⏣⏡⏣⏣⏣⏣⏣⏣⏢w$$v=\underset{u}{\underbrace{⟨v,e_1⟩e_1+\cdots +⟨v,e_m⟩e_m}}+\underset{w}{\underbrace{v-⟨v,e_1⟩e_1+\cdots +⟨v,e_m⟩e_m}}$$So each v∈V${\displaystyle v\in V}$ can be represented by v=u+w${\displaystyle v=u+w}$, which means V=U+W${\displaystyle V=U+W}$. So dim W=dim V-dim U${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}W=\mathrm{d}\mathrm{i}\mathrm{m}V-\mathrm{d}\mathrm{i}\mathrm{m}U}$. Now we should show that ⟨w,ej>=0${\displaystyle ⟨w,e_j⟩=0}$, which means W⊂U⊥${\displaystyle W\subset U^{\perp }}$:a

⟨w,ej>	=⟨v-u,ej>
	=⟨v,ej>-⟨v,ej>⟨ej,ej>
	=0

$$\begin{array}{rl}⟨w,e_j⟩& =⟨v-u,e_j⟩\\ & =⟨v,e_j⟩-⟨v,e_j⟩⟨e_j,e_j⟩\\ & =0\end{array}$$Thus dim V-dim U=dim W⩽dim U⊥${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V-\mathrm{d}\mathrm{i}\mathrm{m}U=\mathrm{d}\mathrm{i}\mathrm{m}W⩽\mathrm{d}\mathrm{i}\mathrm{m}{U}^{\perp }}$. Besides, U∩U⊥={0}${\displaystyle U\cap U^{\perp }=\{0\}}$, it shows dim U+dim U⊥⩽n${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}U+\mathrm{d}\mathrm{i}\mathrm{m}{U}^{\perp }⩽n}$. So there must be(15)dim V=dim U+dim U⊥.$$\mathrm{d}\mathrm{i}\mathrm{m}V=\mathrm{d}\mathrm{i}\mathrm{m}U+\mathrm{d}\mathrm{i}\mathrm{m}{U}^{\perp }.$$Take the both orthonormal basis of U${\displaystyle U}$ and U⊥${\displaystyle U^{\perp }}$, it has right length. So it's a basis of V${\displaystyle V}$.

Theorem144Dimension of the orthogonal complement

Suppose V${\displaystyle V}$ is finite-dimensional and U${\displaystyle U}$ is a subspace of V${\displaystyle V}$. Thendim U⊥=dim V-dim U$$\mathrm{d}\mathrm{i}\mathrm{m}{U}^{\perp }=\mathrm{d}\mathrm{i}\mathrm{m}V-\mathrm{d}\mathrm{i}\mathrm{m}U$$ProofSee (15)

The next result is an important consequence of (14)

Theorem145The orthogonal complement of the orthogonal complement

Suppose U${\displaystyle U}$ is finite-dimensional subspace of V${\displaystyle V}$. ThenU=(U⊥)⊥.$$U=(U^{\perp }{)}^{\perp }.$$

We now define an operator PU${\displaystyle {\mathcal{P}}_U}$ for each finite-dimensional subspace of V${\displaystyle V}$.

Definition146orthogonal projection, PU${\displaystyle P_U}$

Suppose U${\displaystyle U}$ is a finite-dimensional subspace of V${\displaystyle V}$. The orthogonal projection of V${\displaystyle V}$ onto U${\displaystyle U}$ is the operator PU∈L(V)${\displaystyle P_U\in \mathcal{L}(V)}$ defined as follows:For v∈V${\displaystyle v\in V}$, write v=u+w${\displaystyle v=u+w}$, where u∈U${\displaystyle u\in U}$ and w∈U⊥${\displaystyle w\in U^{\perp }}$. Then PUv=u${\displaystyle P_{U}v=u}$.

Theorem147Properties of the orthogonal projection PU${\displaystyle P_U}$

(1) PU=u for every u∈U$$P_U=uforeveryu\in U$$(2) PUw=0 for every w∈U⊥$$P_{U}w=0foreveryw\in U^{\perp }$$(3) range PU=U;$$\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{P}_U=U;$$(4) null PU=U⊥$$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{P}_U=U^{\perp }$$(5) v-PUv∈U⊥$$v-P_{U}v\in U^{\perp }$$(6) P2U=PU$$P_U^2=P_U$$(7) ‖PUv‖⩽‖v‖$$\Vert P_{U}v\Vert ⩽\Vert v\Vert$$(8) for every orthonormal basis e1,⋯,em${\displaystyle e_1,\cdots ,e_m}$ of U${\displaystyle U}$,PUv=⟨v,e1>e1+⋯+⟨v,em>em$$P_{U}v=⟨v,e_1⟩e_1+\cdots +⟨v,e_m⟩e_m$$

Minimization ProblemsThe following problem often arises: given a subspace U${\displaystyle U}$ of V${\displaystyle V}$ and a point v∈V${\displaystyle v\in V}$, find a point u∈U${\displaystyle u\in U}$ such that ‖v-u‖${\displaystyle \Vert v-u\Vert }$ is as small as possible. The next proposition shows that this minimization problem is solved by taking u=PUv${\displaystyle u=P_{U}v}$.

Theorem148Minimizing the distance to a subspace

Suppose U${\displaystyle U}$ is finite-dimensional subspace of V${\displaystyle V}$, v∈V${\displaystyle v\in V}$, and u∈U${\displaystyle u\in U}$. Then‖v-PUv‖⩽‖v-u‖$$\Vert v-P_{U}v\Vert ⩽\Vert v-u\Vert$$Furthermore, the inequality above is an equality if and only if u=PUv.${\displaystyle u=P_{U}v.}$Proof a

‖v-PUv‖2	⩽‖v-PUv‖2+‖PUv-u‖2
	=‖(v-PUv)+(PUv-u)‖2
	=‖v-u‖2

$$\begin{array}{rl}\Vert v-P_{U}v{\Vert }^2& ⩽\Vert v-P_{U}v{\Vert }^2+\Vert P_{U}v-u{\Vert }^2\\ & =\Vert (v-P_{U}v)+(P_{U}v-u){\Vert }^2\\ & =\Vert v-u{\Vert }^2\end{array}$$

Example148.1approximate sin(x)${\displaystyle \sin (x)}$

Find a polynomial u${\displaystyle u}$ with real coefficients and degree at most 5 that approximates sinx${\displaystyle \sin x}$ as well as possible on the interval [-𝜋,𝜋]${\displaystyle [-𝜋,𝜋]}$, in the sense that (16) is as small as possible. (16)𝜋∫-𝜋|sinx-u(x)|2dx$$\underset{-𝜋}{\overset{𝜋}{\int }}|\sin x-u(x){|}^2\mathrm{d}x$$solutionLet v∈CR[-𝜋,𝜋]${\displaystyle v\in C_R[-𝜋,𝜋]}$ be the function defined by v(x)=sinx${\displaystyle v(x)=\sin x}$. Let U${\displaystyle U}$ denote the subspace of CR[-𝜋,𝜋]${\displaystyle C_R[-𝜋,𝜋]}$ consisting of the polynomials with real coefficients and degree at most 5. Our problem can now be reformulated as follows:Find u∈U such that ‖v-u‖ is as small as possible $$\mathrm{F}\mathrm{i}\mathrm{n}\mathrm{d}u\in U\mathrm{s}\mathrm{u}\mathrm{c}\mathrm{h}\mathrm{t}\mathrm{h}\mathrm{a}\mathrm{t}\Vert v-u\Vert \mathrm{i}\mathrm{s}\mathrm{a}\mathrm{s}\mathrm{s}\mathrm{m}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{s}\mathrm{i}\mathrm{b}\mathrm{l}\mathrm{e}$$To compute the solution to our approximation problem, first apply the Gram-Schmidt Procedure to the basis 1,x,x2,x3,x4,x5${\displaystyle 1,x,x^2,x^3,x^4,x^5}$ of U${\displaystyle U}$, producing an orthonormal basis e1,⋯,e6${\displaystyle e_1,\cdots ,e_6}$ of U${\displaystyle U}$. Then, again using the inner product of polynomial, compute PUv${\displaystyle P_{U}v}$ . Doing this shows that u(x)=0.987862x-0155271x3+0.00564312x5$$u(x)=0.987862x-0_{1}55271x^3+0.00564312x^5$$ where the 𝜋's${\displaystyle 𝜋\text{'}s}$ that appear in the exact answer have been replaced with a good decimal approximation.

Fig5:sinx${\displaystyle \sin x}$ and u(x)${\displaystyle u(x)}$

Another good approximation is Taylor polynomialx-x3

3!+x5

5!$$x-\frac{x^3}{3!}+\frac{x^5}{5!}$$To see how good this approximation is, we draw there function

Fig6:sinx${\displaystyle \sin x}$ and u(x)${\displaystyle u(x)}$

Chapter7. Operators on Inner Product SpacesThe deepest results related to inner product spaces deal with the subject to which we now turn — operators on inner product spaces. By exploiting properties of the adjoint, we will develop a detailed description of several important classes of operators on inner product spaces. 7.A Self-Adjoint and Normal OperatorsAdjoints

Definition149adjoint, T*${\displaystyle T^{*}}$

Suppose T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. The adjoint of T${\displaystyle T}$ is the function T*: W→V${\displaystyle T^{*}:W\to V}$ such that⟨Tv,w>=⟨v,T*w>$$⟨Tv,w⟩=⟨v,T^{*}w⟩$$for every v∈V${\displaystyle v\in V}$ and every w∈W${\displaystyle w\in W}$.

To see why this definition make sense, consider ⟨Tv,w>${\displaystyle ⟨Tv,w⟩}$ as a linear functional. Then according to Riesz Representation Theorem, there exist a unique vector in V${\displaystyle V}$ such that this linear functional is given by ⟨v,w'>${\displaystyle ⟨v,w\text{'}⟩}$. Here T*${\displaystyle T^{*}}$ is a linear map from w${\displaystyle w}$ to w'${\displaystyle w\text{'}}$.

More information

The word adjoint has another meaning in linear algebra. In case you encounter the second meaning for adjoint elsewhere, be warned that the two meanings for adjoint are unrelated to each other.

Example149.1Find T*${\displaystyle T^{*}}$

(1) T: R3→R2${\displaystyle T:{\mathbb{R}}^3\to {\mathbb{R}}^2}$T(x1,x2,x3)=(x2+3x3,2x1)$$T(x_1,x_2,x_3)=(x_2+3x_3,2x_1)$$Herea

<(x1,x2,x3), T*(y1,y2)>	=<T(x1,x2,x3), (y1,y2)>
	=y1(x2+3x3)+2x1y2
	=<(x1,x2,x3),(2y2,y1,3y1)>

$$\begin{array}{rl}⟨(x_1,x_2,x_3),T^{*}(y_1,y_2)⟩& =⟨T(x_1,x_2,x_3),(y_1,y_2)⟩\\ & =y_1(x_2+3x_3)+2x_1{y}_2\\ & =⟨(x_1,x_2,x_3),(2y_2,y_1,3y_1)⟩\end{array}$$Thus T*(y1,y2)=(2y2,y1,3y1)${\displaystyle T^{*}(y_1,y_2)=(2y_2,y_1,3y_1)}$□(2) Fix u∈V${\displaystyle u\in V}$ and x∈W${\displaystyle x\in W}$. Define T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$ byTv=<v,u>x$$Tv=⟨v,u⟩x$$Fix w∈W${\displaystyle w\in W}$. Then for every v∈V${\displaystyle v\in V}$ we havea

<v, T*w>	=<Tv,w>
	=<<v,u>x,w>
	=<v,u><x,w>
	=<v,<w,x>u>

$$\begin{array}{rl}⟨v,T^{*}w⟩& =⟨Tv,w⟩\\ & =⟨⟨v,u⟩x,w⟩\\ & =⟨v,u⟩⟨x,w⟩\\ & =⟨v,⟨w,x⟩u⟩\end{array}$$ThusT*w=<w,x>u□$$T^{*}w=⟨w,x⟩u$$

Theorem150The adjoint is a linear map

If T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$, then T*∈L(W,V)${\displaystyle T^{*}\in \mathcal{L}(W,V)}$.

Theorem151Properties of the adjoint

(1) (S+T)*=S*+T*${\displaystyle (S+T{)}^{*}=S^{*}+T^{*}}$ for all S,T∈L(V,W)${\displaystyle S,T\in \mathcal{L}(V,W)}$;(2) (𝜆T)*=⏨𝜆T*${\displaystyle (𝜆T{)}^{*}=\overline{𝜆}{T}^{*}}$ for all 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$ and T∈L(V,W);${\displaystyle T\in \mathcal{L}(V,W);}$(3) (T*)*=T${\displaystyle (T^{*}{)}^{*}=T}$ for all T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$;a

<w,(T*)*v>	=<T*w,v>=<v,T*w>
	=<Tv,w>=<w,Tv>

$$\begin{array}{rl}⟨w,(T^{*}{)}^{*}v⟩& =⟨T^{*}w,v⟩=⟨v,T^{*}w⟩\\ & =⟨Tv,w⟩=⟨w,Tv⟩\end{array}$$(4) I*=I${\displaystyle {\mathbb{I}}^{*}=\mathbb{I}}$(5) (ST)*=T*S*${\displaystyle (ST{)}^{*}=T^{*}{S}^{*}}$ for all T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$ and S∈L(W,U)${\displaystyle S\in \mathcal{L}(W,U)}$a

<v,(ST)*u>	=<(ST)v,u>
	=<S(Tv),u>
	=<Tv,S*u>
	=<v,T*S*u>

$$\begin{array}{rl}⟨v,(ST{)}^{*}u⟩& =⟨(ST)v,u⟩\\ & =⟨S(Tv),u⟩\\ & =⟨Tv,S^{*}u⟩\\ & =⟨v,T^{*}{S}^{*}u⟩\end{array}$$

Theorem152Null space and range T*${\displaystyle T^{*}}$

Suppose T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$, Then(1) null T*=(range T)⊥${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{*}=(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T{)}^{\perp }}$;(2) range T*=(null T)⊥${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^{*}=(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T{)}^{\perp }}$;(3) null T=(range T*)⊥${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T=(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^{*}{)}^{\perp }}$; (Replace T${\displaystyle T}$ with T*${\displaystyle T^{*}}$ in (1))(4) range T=(null T*)⊥${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T=(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{*}{)}^{\perp }}$

Definition153conjugate transpose

The conjugate transpose of an m${\displaystyle m}$-by-n${\displaystyle n}$ matrix is the n${\displaystyle n}$-by-m${\displaystyle m}$ matrix obtained by interchanging the rows and columns and then taking the complex conjugate of each entry.

The next result shows how to compute the matrix of T*${\displaystyle T^{*}}$ from the matrix of T${\displaystyle T}$.

Theorem154The matrix of T*${\displaystyle T^{*}}$

Let T∈L(V,W)${\displaystyle T\in \mathcal{L}(V,W)}$. Suppose e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ is an orthonormal basis of V${\displaystyle V}$ and f1,⋯,fm${\displaystyle f_1,\cdots ,f_m}$ is an orthonormal basis of W${\displaystyle W}$. ThenM(T*,(f1,⋯,fm), (e1,⋯,em))$$\mathcal{M}(T^{*},(f_1,\cdots ,f_m),(e_1,\cdots ,e_m))$$is the conjugate transpose of M(T,(e1,⋯,en), (f1,⋯,fm).$$\mathcal{M}(T,(e_1,\cdots ,e_n),(f_1,\cdots ,f_m).$$Proof

Recall that we obtain the kth${\displaystyle k^{\mathrm{t}\mathrm{h}}}$ column of M(T)${\displaystyle \mathcal{M}(T)}$ by writing Tek${\displaystyle T{e}_k}$ as a linear combination of the fj${\displaystyle f_j}$'s; the scalars used in this linear combination then become the k${\displaystyle k}$th column of M(T)${\displaystyle \mathcal{M}(T)}$. So we have

a A11 A12 A21 A22 A31 A32 a v1 v2 v3 =a A11u1+A12u2 A21u1+A22u2 A31u1+A32u2 a u1 u2 It shows how the 2th component in U${\displaystyle U}$ reacts with the linear map, and contributes to the3th component in V${\displaystyle V}$1th row gives the coefficient on the 1thbasis of V${\displaystyle V}$1th column shows how the 1th basis of U${\displaystyle U}$ corresponds to the linear map

Tek=<Tek,f1>f1+⋯+<Tek,fm>fmM(T)j,k=<Tek,fj> $$\begin{array}{c}T{e}_k=⟨T{e}_k,f_1⟩f_1+\cdots +⟨T{e}_k,f_m⟩f_m\\ \mathcal{M}(T{)}_{j,k}=⟨T{e}_k,f_j⟩\\ \end{array}$$Replacing T${\displaystyle T}$ with T*${\displaystyle T^{*}}$ and interchanging the roles played by the e'${\displaystyle e\text{'}}$s and f${\displaystyle f}$'s, we see that the entry in row j${\displaystyle j}$, column k${\displaystyle k}$, of M(T*)${\displaystyle \mathcal{M}\left(T^{*}\right)}$ is <T*fk,ej>${\displaystyle ⟨T^{*}{f}_k,e_j⟩}$, which equals <fk,Tej>${\displaystyle ⟨f_k,T{e}_j⟩}$ ${\displaystyle }$, which equals ⏨⏨⏨⏨<Tej,fk>${\displaystyle \overline{⟨T{e}_j,f_k⟩}}$, which equals the complex conjugate of the entry in row k${\displaystyle k}$, column j${\displaystyle j}$, of M(T)${\displaystyle \mathcal{M}(T)}$.

Caution

Remember that the result below applies only when we are dealing with orthonormal bases. With respect to nonorthonormal bases, the matrix of T*${\displaystyle T^{*}}$ does not necessarily equal the conjugate transpose of the matrix of T${\displaystyle T}$.

Definition155self-adjoint

An operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is called self-adjoint if T=T*${\displaystyle T=T^{*}}$. In other words, T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is self-adjoint if and only if <Tv,w>=<v,Tw>$$⟨Tv,w⟩=⟨v,Tw⟩$$for all v,w∈V${\displaystyle v,w\in V}$.

More Information

It's also called Hermitian.

If F=R${\displaystyle \mathbb{F}=\mathbb{R}}$, then by definition every eigenvalues is real, so the next result is interesting only when F=C${\displaystyle \mathbb{F}=\mathbb{C}}$.

Theorem156Eigenvalues of self-adjoint operators are real

Every eigenvalue of a self-adjoint operator is real.Proof Suppose Tv=𝜆v${\displaystyle Tv=𝜆v}$, then a

<Tv,v>	=<v,T*v>
<𝜆v,v>	=<v,𝜆v>
𝜆<v,v>	=⏨𝜆<v,v>

$$\begin{array}{rl}⟨Tv,v⟩& =⟨v,T^{*}v⟩\\ ⟨𝜆v,v⟩& =⟨v,𝜆v⟩\\ 𝜆⟨v,v⟩& =\overline{𝜆}⟨v,v⟩\end{array}$$Thus 𝜆=⏨𝜆${\displaystyle 𝜆=\overline{𝜆}}$□

The next result is false for real inner product spaces. As an example, consider the operator T∈L(R2)${\displaystyle T\in \mathcal{L}\left({\mathbb{R}}^2\right)}$that is counterclockwise rotation of 90°${\displaystyle 90°}$ around the origin; thus T(x,y)=(-y,x)${\displaystyle T(x,y)=(-y,x)}$. Obviously Tv${\displaystyle Tv}$ is orthogonal to v${\displaystyle v}$ for every v∈R2${\displaystyle v\in {\mathbb{R}}^2}$, even though T≠0${\displaystyle T\ne 0}$

Theorem157Over C, Tv${\displaystyle \mathbb{C},Tv}$ is orthogonal to all v${\displaystyle v}$ only for the 0${\displaystyle 0}$ operator

Suppose V${\displaystyle V}$ is a complex inner product space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Suppose(17)<Tv,v>=0$$⟨Tv,v⟩=0$$for all v∈V${\displaystyle v\in V}$. Then T=0${\displaystyle T=0}$.ProofEvery <Tu,w>${\displaystyle ⟨Tu,w⟩}$ can be computed as some analogical terms of equ(17).<Tu,w>=<T(u+w),u+w>-<T(u-w),u-w>

4+<T(u+ⅈw),u+ⅈw>-<T(u-ⅈw),u-ⅈw>

4ⅈ$$⟨Tu,w⟩=\frac{⟨T(u+w),u+w⟩-⟨T(u-w),u-w⟩}{4}+\frac{⟨T(u+iw),u+iw⟩-⟨T(u-iw),u-iw⟩}{4}i$$So if equ (17) holds, Tu${\displaystyle Tu}$ is orthonormal to all the vector in space V${\displaystyle V}$.

The next result is false for real inner product spaces, as shown by considering any operator on a real inner product space that is not self-adjoint. And this theorem also appears in quantum physics.

Theorem158Over C${\displaystyle \mathbb{C}}$, <Tv,v>${\displaystyle ⟨Tv,v⟩}$is real for all v${\displaystyle v}$ only for self-adjoint operators

Suppose V${\displaystyle V}$ is a complex inner product space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then T${\displaystyle T}$ is self-adjoint if and only if<Tv,v>∈R.$$⟨Tv,v⟩\in \mathbb{R}.$$ProofLet v∈V.${\displaystyle v\in V.}$ Thena

<Tv,v>-⏨⏨⏨⏨<Tv,v>	=<Tv,v>-<v,Tv>
	=<Tv,v>-<T*v,v>
	=<(T-T*)v,v>

$$\begin{array}{rl}⟨Tv,v⟩-\overline{⟨Tv,v⟩}& =⟨Tv,v⟩-⟨v,Tv⟩\\ & =⟨Tv,v⟩-⟨T^{*}v,v⟩\\ & =⟨(T-T^{*})v,v⟩\end{array}$$If <Tv,v>∈R${\displaystyle ⟨Tv,v⟩\in \mathbb{R}}$ for every v∈V${\displaystyle v\in V}$, then T=T*${\displaystyle T=T^{*}}$.

On a real inner product space V${\displaystyle V}$, a nonzero operator T${\displaystyle T}$ might satisfy <Tv,v>=0${\displaystyle ⟨Tv,v⟩=0}$ for all v∈V${\displaystyle v\in V}$. However, the next result shows that this cannot happen for a self-adjoint operator.

Theorem159If T=T*${\displaystyle T=T^{*}}$ and <Tv,v>=0${\displaystyle ⟨Tv,v⟩=0}$ for all v${\displaystyle v}$, then T=0${\displaystyle T=0}$

Note that this theorem also holds on real inner product space.Proof We use another transformation:<Tu,w>=<T(u+w),u+w>-<T(u-w),u-w>

4;$$⟨Tu,w⟩=\frac{⟨T(u+w),u+w⟩-⟨T(u-w),u-w⟩}{4};$$this is correct only when<Tw,u>=<w,Tu>=<Tu,w>$$⟨Tw,u⟩=⟨w,Tu⟩=⟨Tu,w⟩$$

Normal Operators

Definition160normal

An operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ on an inner product space is called normal if it commutes with its adjoint. In other words,T T*=T*T$$T{T}^{*}=T^{*}T$$

Obviously, every self-adjoint operator is normal.

Theorem161T${\displaystyle T}$ is normal if and only if ‖Tv‖=‖T*v‖${\displaystyle \Vert Tv\Vert =\Vert T^{*}v\Vert }$ for all v${\displaystyle v}$

Proof a

T is normal	⟺TT*-T*T=0
	⟺<(T*T-TT*)v,v>=0
	⟺<T*Tv,v>=<TT*v,v>
	⟺<Tv,Tv>=<T*v,T*v>
	⟺‖Tv‖2=‖T*v‖2

$$\begin{array}{rl}T\mathrm{i}\mathrm{s}\mathrm{n}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}& ⟺T{T}^{*}-T^{*}T=0\\ & ⟺⟨(T^{*}T-T{T}^{*})v,v⟩=0\\ & ⟺⟨T^{*}Tv,v⟩=⟨T{T}^{*}v,v⟩\\ & ⟺⟨Tv,Tv⟩=⟨T^{*}v,T^{*}v⟩\\ & ⟺\Vert Tv{\Vert }^2=\Vert T^{*}v{\Vert }^2\end{array}$$

It can be proved that the eigenvalues of the adjoint of each operator are equal (as a set) to the complex conjugates of the eigenvalues of the operator. But an operator and its adjoint may have different eigenvectors. However, a normal operator and its adjoint have the same eigenvectors.

Theorem162For T${\displaystyle T}$ normal, T${\displaystyle T}$ and T*${\displaystyle T^{*}}$ have the same eigenvectors

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is normal and v∈V${\displaystyle v\in V}$ is an eigenvector of T${\displaystyle T}$ with eigenvalue 𝜆${\displaystyle 𝜆}$. Then v ${\displaystyle v}$is also an eigenvector of T*${\displaystyle T^{*}}$ with eigenvalue ⏨𝜆${\displaystyle \overline{𝜆}}$.Proof Suppose Tv=𝜆v${\displaystyle Tv=𝜆v}$. Thus (T-𝜆I)v=0$$(T-𝜆\mathbb{I})v=0$$Because T-𝜆I${\displaystyle T-𝜆\mathbb{I}}$ is also normal, so using last theorem, we have‖(T-𝜆I)v‖=‖(T-𝜆I)*v‖=0$$\Vert (T-𝜆\mathbb{I})v\Vert =\Vert (T-𝜆\mathbb{I}{)}^{*}v\Vert =0$$So a vector has 0 norm when it's a 0${\displaystyle 0}$(T*-⏨𝜆I)v=0$$(T^{*}-\overline{𝜆}\mathbb{I})v=0$$□

7.B The spectral TheoremThe nicest operators on V${\displaystyle V}$ are those for which there is an orthonormal basis of V${\displaystyle V}$ with respect to which the operator has a diagonal matrix. (We are not dealing with diagonalizing. We are dealing with a more special case: the eigenvector are orthonormal). These are precisely the operators T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ such that there is an orthonormal basis of V${\displaystyle V}$ consisting of eigenvectors of T${\displaystyle T}$. Our goal in this section is to prove the Spectral Theorem, which characterizes these operators as the normal operators when F=C${\displaystyle \mathbb{F}=\mathbb{C}}$ and as the self-adjoint operators when F=R${\displaystyle \mathbb{F}=\mathbb{R}}$. The Spectral Theorem is probably the most useful tool in the study of operators on inner product spaces. Because the conclusion of Spectral Theorem depends on F${\displaystyle \mathbb{F}}$, we will break the Spectral Theorem into two pieces, called the Complex Spectral Theorem and the Real Spectral Theorem. As is often the case in linear algebra, complex vector spaces are easier to deal with than real vector spaces. Thus we present the Complex Spectral Theorem first. The Complex Spectral TheoremThe key part of the CST states that if F=C${\displaystyle \mathbb{F}=\mathbb{C}}$ and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is normal, then T${\displaystyle T}$ has a diagonal matrix with respect to some orthonormal basis of V${\displaystyle V}$.

Theorem163Complex Spectral Theorem

Suppose F=C${\displaystyle \mathbb{F}=\mathbb{C}}$ and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then the following are equivalent:(1) T${\displaystyle T}$ is normal(2) V${\displaystyle V}$ has an orthonormal basis consisting of eigenvectors of T${\displaystyle T}$.(3) T${\displaystyle T}$ has a diagonal matrix with respect to some orthonormal basis of V${\displaystyle V}$.Proof We have already shown (2)⟺(3)${\displaystyle (2)⟺(3)}$. First suppose (3) holds, so T${\displaystyle T}$ has a diagonal matrix. The matrix of T*${\displaystyle T^{*}}$ is obtained by taking the conjugate transpose of the matrix of T${\displaystyle T}$; hence T*${\displaystyle T^{*}}$ also has a diagonal matrix. Any diagonal matrix commute; thus T${\displaystyle T}$ is normal. Now suppose (1) holds, so T${\displaystyle T}$ is normal. By Schur's Theorem there is an orthonormal basis e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$of V${\displaystyle V}$ with respect to which T${\displaystyle T}$ has an upper-triangular matrix. Thus we can writeM(T)=a

a1,1	⋯	a1,n
	⋱	⋮
0		an,n

$$\mathcal{M}(T)=\left(\begin{array}{ccc}{a}_{1,1}& \cdots & a_{1,n}\\ & \ddots & ⋮\\ 0& & a_{n,n}\end{array}\right)$$ We will show that this matrix is actually a diagonal matrix. We see from the matrix above that‖Te1‖2=||a1,1||2$$\Vert T{e}_1{\Vert }^2=|a_{1,1}{|}^2$$and‖T*e1‖2=||a1,1||2+||a1,2||2+⋯+||a1,n||2.$$\Vert T^{*}{e}_1{\Vert }^2=|a_{1,1}{|}^2+|a_{1,2}{|}^2+\cdots +|a_{1,n}{|}^2.$$Because T${\displaystyle T}$ is normal, so ‖Te1‖=‖T*e1‖${\displaystyle \Vert T{e}_1\Vert =\Vert T^{*}{e}_1\Vert }$. Thus the two equations give the result:||a1,2||=⋯=||a1,n||=0$$|a_{1,2}|=\cdots =|a_{1,n}|=0$$Now we have‖Te2‖2=||a2,2||2‖T*e2‖2=||a2,2||2+||a2,3||2+⋯$$\begin{array}{c}\Vert T{e}_2{\Vert }^2=|a_{2,2}{|}^2\\ \Vert T^{*}{e}_2{\Vert }^2=|a_{2,2}{|}^2+|a_{2,3}{|}^2+\cdots \end{array}$$So we have result||a2,3||=⋯=||a2,n||=0$$|a_{2,3}|=\cdots =|a_{2,n}|=0$$Continue in this fashion, we see that all the nondiagonal entries in the matrix equal 0.□

The Real Spectral TheoremThis theorem is the kernel of the spectral theorem:

Theorem164Invertible quadratic expressions

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is self-adjoint and b,c∈R${\displaystyle b,c\in \mathbb{R}}$ are such that b2<4c${\displaystyle b^2<4c}$. ThenT2+bT+cI$$T^2+bT+cI$$is invertible.Proof Let v${\displaystyle v}$ be a nonzero vector in V${\displaystyle V}$.a

<(T2+bT+cI)v,v>	=<T2v,v>+b<Tv,v>+c<v,v>
	=<Tv,Tv>+b<Tv,v>+c‖v‖2
	⩾‖Tv‖2+||b||‖Tv‖ ‖v‖+c‖v‖2
	>0

$$\begin{array}{rl}⟨(T^2+bT+c\mathbb{I})v,v⟩& =⟨T^{2}v,v⟩+b⟨Tv,v⟩+c⟨v,v⟩\\ & =⟨Tv,Tv⟩+b⟨Tv,v⟩+c\Vert v{\Vert }^2\\ & ⩾\Vert Tv{\Vert }^2+|b|\Vert Tv\Vert \Vert v\Vert +c\Vert v{\Vert }^2\\ & >0\end{array}$$It implies (T2+bT+cI)v${\displaystyle (T_{}^2+bT+c\mathbb{I})v}$ orthonormal with v${\displaystyle v}$. so (T2+bT+cI)v≠0${\displaystyle (T_{}^2+bT+c\mathbb{I})v\ne 0}$. So null space is only {0}, it is invertible.

We know that every operator, self-adjoint or not, on a finite-dimensional nonzero complex vector space has an eigenvalue.

Theorem165Self-adjoint operators have eigenvalues

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is self-adjoint Then T${\displaystyle T}$ has an eigenvalue.Proof We can assume that V${\displaystyle V}$ is a real inner product space, as we have already noted. Let n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$ and choose v∈V${\displaystyle v\in V}$ with v≠0${\displaystyle v\ne 0}$. Thenv,Tv,⋯,Tnv$$v,Tv,\cdots ,T^{n}v$$cannot be linearly independent. 0=a0v+a1Tv+⋯+anTnv.$$0=a_{0}v+a_{1}Tv+\cdots +a_n{T}^{n}v.$$Make the a${\displaystyle a}$'s coefficients of a polynomial, which can be written in factored form asa0+a1x+⋯+anxn=c(x2+b1x+c1)⋯(x2+bMx+cM)(x-𝜆1)⋯(x-𝜆m)$$a_0+a_{1}x+\cdots +a_n{x}^n=c(x^2+b_{1}x+c_1)\cdots (x^2+b_{M}x+c_M)(x-{𝜆}_1)\cdots (x-{𝜆}_m)$$where c${\displaystyle c}$ is a nonzero real number, each bj,cj${\displaystyle b_j,c_j}$ and 𝜆j${\displaystyle {𝜆}_j}$ is real, each b2j${\displaystyle b_j^2}$ is less than 4cj${\displaystyle 4c_j}$, m+M⩾1${\displaystyle m+M⩾1}$, and the equation holds for all real x${\displaystyle x}$. We then havea

0	=a0v+⋯anTnv
	=(a0I+a1T+⋯+anTn)v
	=c(T2+b1T+c1I)⋯(T2+bMT+cMI)(T-𝜆1I)⋯(T-𝜆mI)v

$$\begin{array}{rl}0& =a_{0}v+\cdots a_n{T}^{n}v\\ & =(a_0\mathbb{I}+a_{1}T+\cdots +a_n{T}^n)v\\ & =c(T^2+b_{1}T+c_1\mathbb{I})\cdots (T^2+b_{M}T+c_M\mathbb{I})(T-{𝜆}_1\mathbb{I})\cdots (T-{𝜆}_m\mathbb{I})v\end{array}$$By last Theorem, each T2+b1T+c1I${\displaystyle T^2+b_{1}T+c_1\mathbb{I}}$ is invertible. Recall also that c≠0${\displaystyle c\ne 0}$. Thus the equation above implies that m>0${\displaystyle m>0}$ and0=(T-𝜆1I)⋯(T-𝜆mI)v$$0=(T-{𝜆}_1\mathbb{I})\cdots (T-{𝜆}_m\mathbb{I})v$$Hence T-𝜆jI${\displaystyle T-{𝜆}_j\mathbb{I}}$ is note injective for at least one j${\displaystyle j}$. In other words, T${\displaystyle T}$ has an eigenvalue.

The next result shows that if U${\displaystyle U}$ is a subspace of V${\displaystyle V}$ that is invariant under a self-adjoint operator T${\displaystyle T}$, then U⊥${\displaystyle U^{\perp }}$ is also invariant under T${\displaystyle T}$.

Theorem166Self-adjoint operators and invariant subspaces

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is self-adjoint U${\displaystyle U}$ is a subspace of V${\displaystyle V}$ that is invariant under T${\displaystyle T}$. Then(1) U⊥${\displaystyle U^{\perp }}$ is invariant under T${\displaystyle T}$(2) T|U∈L(U)${\displaystyle T{|}_U\in \mathcal{L}(U)}$ is self-adjoint(3) T|U⊥∈L(U⊥)${\displaystyle T{|}_{U^{\perp }}\in \mathcal{L}\left(U^{\perp }\right)}$is self-adjoint Proof U${\displaystyle U}$ is invariant means that <Tu,v>=0${\displaystyle ⟨Tu,v⟩=0}$ for v${\displaystyle v}$ not in U${\displaystyle U}$, which is actually a vector of U⊥${\displaystyle U^{\perp }}$. So we have similarly<Tv,u>=<v,Tu>=0$$⟨Tv,u⟩=⟨v,Tu⟩=0$$To prove (2) and (3)<(T|U)u1,u2>=<Tu1,u2>=<u1,Tu2>=<u1,(T|U)u2>$$⟨(T{|}_U)u_1,u_2⟩=⟨T{u}_1,u_2⟩=⟨u_1,T{u}_2⟩=⟨u_1,(T{|}_U)u_2⟩$$

Theorem167Real Spectral Theorem

Suppose F=R${\displaystyle \mathbb{F}=\mathbb{R}}$ and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$.Then the following are equivalent(1) T${\displaystyle T}$ is self-adjoint(2) V${\displaystyle V}$ has an orthonormal basis consisting of eigenvectors of T${\displaystyle T}$.(3) T${\displaystyle T}$ has a diagonal matrix with respect to some orthonormal basis of V${\displaystyle V}$.Proof First suppose (3) holds, so T${\displaystyle T}$ has a diagonal matrix. Hence T=T*${\displaystyle T=T^{*}}$. We now prove (1) implies (2) by induction on dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V}$. It's easy to show if dim V=1${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V=1}$, it holds. If dim V⩾2${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V⩾2}$, because V${\displaystyle V}$ is self-adjoint, so it must have a eigenvector, namely, u${\displaystyle u}$. Let U=span(u)${\displaystyle U=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(u)}$, which is invariant under T${\displaystyle T}$. Thus U⊥${\displaystyle U^{\perp }}$ is also invariant under T|U⊥${\displaystyle T{|}_{U^{\perp }}}$ and has an orthonormal basis consisting of eigenvectors of T|U⊥${\displaystyle T{|}_{U^{\perp }}}$. Append this basis with u/‖u‖${\displaystyle u/\Vert u\Vert }$, we get an orthonormal basis of V${\displaystyle V}$.

7.C Positive Operators and IsometriesPositive Operators

Definition168Positive operator

An operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is called positive if T${\displaystyle T}$ is self-adjoint and(18)<Tv,v>⩾0$$⟨Tv,v⟩⩾0$$

Definition169square root

An operator R${\displaystyle R}$ is called a square root of an operator T${\displaystyle T}$ if R2=T${\displaystyle R^2=T}$

The characterizations of the positive operators in the next result correspond to characterizations of the nonnegative numbers among C${\displaystyle \mathbb{C}}$. Specifically, a complex number z${\displaystyle z}$ is nonnegative if and only if it has a nonnegative square root. Also, z${\displaystyle z}$ is nonnegative if and only if it has a real square root, corresponding to condition (4). Finally, z${\displaystyle z}$ is nonnegative if and only if there exists a complex number w${\displaystyle w}$ such that z=⏨ww${\displaystyle z=\overline{w}w}$, corresponding to condition (5)

Theorem170Characterization of positive operators

Let T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then the following are equivalent:(1) T${\displaystyle T}$ is positive;(2) T${\displaystyle T}$ is self-adjoint and all the eigenvalues of T${\displaystyle T}$ are nonnegative(3) T${\displaystyle T}$ has a positive square root(4) T${\displaystyle T}$ has a self-adjoint square root(5) there exists an operator R∈L(V)${\displaystyle R\in \mathcal{L}(V)}$ such that T=R*R${\displaystyle T=R^{*}R}$ProofWe will prove that (1)⇒(2)⇒(3)⇒(4)⇒(5)⇒(1)${\displaystyle (1)\Rightarrow (2)\Rightarrow (3)\Rightarrow (4)\Rightarrow (5)\Rightarrow (1)}$.First suppose (1) holds, then T${\displaystyle T}$ is self-adjoint. According to Spectral theorem, it has a diagonalized matrix respected to some basis. So if we need (18) holds, (2) must hold. Then (3) just need a matrix R${\displaystyle R}$ with all the entries the square root of T${\displaystyle T}$. (4) and (5) just use the same matrix R${\displaystyle R}$ as (3) does. If (5) holds, then<Tv,v>=<R*Rv,v>=<Rv,Rv>⩾0$$⟨Tv,v⟩=⟨R^{*}Rv,v⟩=⟨Rv,Rv⟩⩾0$$

Theorem171Each positive operator has only one positive square root

ProofLet R${\displaystyle R}$ be a positive square root of T${\displaystyle T}$. We will prove that Rv=𝜆v${\displaystyle Rv=\sqrt{𝜆}v}$. (with supposition Tv=𝜆v${\displaystyle Tv=𝜆v}$) This will imply that the behavior of R${\displaystyle R}$ on the eigenvectors of T${\displaystyle T}$ is uniquely determined. Because there is a basis of V${\displaystyle V}$ consisting of eigenvectors of T${\displaystyle T}$, this will imply that R${\displaystyle R}$ is uniquely determined. To prove Rv=𝜆v${\displaystyle Rv=\sqrt{𝜆}v}$, note that the Spectral Theorem asserts that there is an orthonormal basis e1,⋯,em${\displaystyle e_1,\cdots ,e_m}$ of V${\displaystyle V}$ consisting of eigenvectors of R${\displaystyle R}$. Because R${\displaystyle R}$ is a positive operator, all its eigenvalues are nonnegative. Thus there exist nonnegative numbers 𝜆1,⋯,𝜆n${\displaystyle {𝜆}_1,\cdots ,{𝜆}_n}$ such that Rej=𝜆jej${\displaystyle R{e}_j=\sqrt{{𝜆}_j}{e}_j}$ for j=1,⋯,n${\displaystyle j=1,\cdots ,n}$.

Isometries (unitary operator)

Theorem172isometry

An operator S∈L(V)${\displaystyle S\in \mathcal{L}(V)}$ is called an isometry if it preserves norms.‖Sv‖=‖v‖$$\Vert Sv\Vert =\Vert v\Vert$$These statements are equivalent:(1) S${\displaystyle S}$ is an isometry(2) <Su,Sv>=<u,v>${\displaystyle ⟨Su,Sv⟩=⟨u,v⟩}$ for all u,v∈V${\displaystyle u,v\in V}$;(3) Se1,⋯,Sen${\displaystyle S{e}_1,\cdots ,S{e}_n}$ is orthonormal for every orthonormal list of vectors e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ in V${\displaystyle V}$;(4) there exists an orthonormal basis e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ of V${\displaystyle V}$ such that Se1,⋯,Sen${\displaystyle S{e}_1,\cdots ,S{e}_n}$ is orthonormal(5) S*S=I${\displaystyle S^{*}S=\mathbb{I}}$(6) SS*=I${\displaystyle S{S}^{*}=\mathbb{I}}$(7) S*${\displaystyle S^{*}}$ is an isometry(8) S${\displaystyle S}$ is invertible and S-1=S*${\displaystyle S^{-1}=S^{*}}$.

Proof (1) and (2) is equivalent by definition. So (3) holds. (4) holds as well. Now suppose (4) holds. Let e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ be an orthonormal basis of V${\displaystyle V}$ such that Se1,⋯,Sen${\displaystyle S{e}_1,\cdots ,S{e}_n}$ is orthonormal. Thus<Sej,Sek>=<S*Sej,ek>=<ej,ek>$$⟨S{e}_j,S{e}_k⟩=⟨S^{*}S{e}_j,e_k⟩=⟨e_j,e_k⟩$$All vectors u,v∈V${\displaystyle u,v\in V}$ can be written as linear combinations of e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$, and thus the equation above implies that <S*Su,v>=<u,v>${\displaystyle ⟨S^{*}Su,v⟩=⟨u,v⟩}$. Hence <(S*S-I)u,v>=0${\displaystyle ⟨(S^{*}S-\mathbb{I})u,v⟩=0}$, S*S=I${\displaystyle S^{*}S=\mathbb{I}}$. (5) holds. Thus ‖Sv‖2=‖S*v‖2=<SS*v,v>=‖v‖2$$\Vert Sv{\Vert }^2=\Vert S^{*}v{\Vert }^2=⟨S{S}^{*}v,v⟩=\Vert v{\Vert }^2$$(6) holds. (7) holds. (8) holds. If (8) holds, <Sv,Sv>=<S*Sv,v>=<v,v>${\displaystyle ⟨Sv,Sv⟩=⟨S^{*}Sv,v⟩=⟨v,v⟩}$. Thus (1) holds.

Theorem173Description of isometries when F=C${\displaystyle \mathbb{F}=\mathbb{C}}$

Suppose V${\displaystyle V}$ is a complex inner product space and S∈L(V)${\displaystyle S\in \mathcal{L}(V)}$. Then the following are equivalent:(1) S${\displaystyle S}$ is an isometry(2) There is an orthonormal basis of V${\displaystyle V}$ consisting of eigenvectors of S${\displaystyle S}$ whose corresponding eigenvalues all have absolute value 1.ProofIt's easy to show that (2) implies (1). To show the other direction, suppose (1) holds.So S ${\displaystyle S}$is an isometry. By the Complex Spectral Theorem, there is an orthonormal basis e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ of V${\displaystyle V}$ consisting of eigenvectors of S${\displaystyle S}$. For j∈{1,⋯,n}${\displaystyle j\in \{1,\cdots ,n\}}$, let 𝜆j${\displaystyle {𝜆}_j}$ be the eigenvalue corresponding to ej${\displaystyle e_j}$. Then||𝜆j||=‖𝜆jej‖=‖Sej‖=‖ej‖=1$$|{𝜆}_j|=\Vert {𝜆}_j{e}_j\Vert =\Vert S{e}_j\Vert =\Vert e_j\Vert =1$$Thus each eigenvalue of S${\displaystyle S}$ has absolute value 1.

7.D Polar Decomposition and Singular Value DecompositionPolar decompositionWe have found the analogy between C${\displaystyle \mathbb{C}}$ and L(V)${\displaystyle \mathcal{L}(V)}$. Continuing with our analogy, note that each complex number z${\displaystyle z}$ except 0 can be written in the formz=(z

||z||)||z||=(z

||z||)⏨zz$$z=\left(\frac{z}{|z|}\right)|z|=\left(\frac{z}{|z|}\right)\sqrt{\overline{z}z}$$Our analogy leads us to guess that each operator T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ can be written as an isometry times T*T${\displaystyle \sqrt{T^{*}T}}$.

Definition174T${\displaystyle \sqrt{T}}$

If T${\displaystyle T}$ is a positive operator, then T ${\displaystyle \sqrt{T}}$denotes the unique positive square root of T${\displaystyle T}$.

Note that T*T${\displaystyle T^{*}T}$ is a positive operator for every T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$, (<T*Tv,v>⩾0)${\displaystyle (⟨T^{*}Tv,v⟩⩾0)}$, so the theorem is reasonable.

Theorem175Polar Decomposition

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then there exists an isometry S∈L(V)${\displaystyle S\in \mathcal{L}(V)}$ such thatT=ST*T.$$T=S\sqrt{T^{*}T.}$$ProofIf v∈V${\displaystyle v\in V}$ , thena

‖Tv‖2	=<Tv,Tv>
	=<T*Tv,v>
	=<T*Tv,T*Tv>
	=‖T*Tv‖2

$$\begin{array}{rl}\Vert Tv{\Vert }^2& =⟨Tv,Tv⟩\\ & =⟨T^{*}Tv,v⟩\\ & =⟨\sqrt{T^{*}T}v,\sqrt{T^{*}T}v⟩\\ & =\Vert \sqrt{T^{*}T}v{\Vert }^2\end{array}$$So T${\displaystyle T}$ and T*T${\displaystyle \sqrt{T^{*}T}}$ preserve the norm. Thus we only need to construct a isometry between them:(19)S1(T*Tv):=Tv$$S_1\left(\sqrt{T^{*}T}v\right):=Tv$$ First we must check that S1${\displaystyle S_1}$ is well defined. To do this, suppose v1,v2∈V${\displaystyle v_1,v_2\in V}$ are such that T*Tv1=T*Tv2${\displaystyle \sqrt{T^{*}T}{v}_1=\sqrt{T^{*}T}{v}_2}$. For the definition given by (19), we must show that Tv1=Tv2${\displaystyle T{v}_1=T{v}_2}$a

‖Tv1-Tv2‖	=‖T(v1-v2)‖
	=‖T*T(v1-v2)‖
	=‖T*Tv1-T*Tv2‖
	=0

$$\begin{array}{rl}\Vert T{v}_1-T{v}_2\Vert & =\Vert T(v_1-v_2)\Vert \\ & =\Vert \sqrt{T^{*}T}(v_1-v_2)\Vert \\ & =\Vert \sqrt{T^{*}T}{v}_1-\sqrt{T^{*}T}{v}_2\Vert \\ & =0\end{array}$$Thus Tv1=Tv2${\displaystyle T{v}_1=T{v}_2}$. However, this just define the operator in a subspace. What about S${\displaystyle S}$ on the outside of rangeT*T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\sqrt{T^{*}T}}$ ? In particular, S${\displaystyle S}$ is injective. Thus form the Fundamental Theorem of Linear Map we havedim rangeT*T=dim range Tdim(range T*T)⊥=dim(range T)⊥$$\begin{array}{c}\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\sqrt{T^{*}T}=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T\\ \mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\sqrt{T^{*}T}{)}^{\perp }=\mathrm{d}\mathrm{i}\mathrm{m}(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T{)}^{\perp }\end{array}$$So find two basis for (range T*T)⊥ and (range T)⊥${\displaystyle (\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\sqrt{T^{*}T}{)}^{\perp }\mathrm{a}\mathrm{n}\mathrm{d}(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T{)}^{\perp }}$, define a S2${\displaystyle S_2}$, preserving the coefficients of the basis. Then let S${\displaystyle S}$ equal S1${\displaystyle S_1}$ on rangeT*T${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\sqrt{T^{*}T}}$ and equal S2${\displaystyle S_2}$ on (rangeT*T)⊥${\displaystyle (\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}\sqrt{T^{*}T}{)}^{\perp }}$

Singular Value Decomposition

Definition176singular values

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. The singular values of T${\displaystyle T}$ are the eigenvalues of T*T${\displaystyle \sqrt{T^{*}T}}$, with each eigenvalue 𝜆${\displaystyle 𝜆}$ repeated dim E(𝜆,T*T)${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathbb{E}(𝜆,\sqrt{T^{*}T})}$ times. Actually, we can just compute them by computing eigenvalues of T*T${\displaystyle T^{*}T}$. Singular values are the square root of eigenvalues.

Example176.1singular values

Define T∈L(F4)${\displaystyle T\in \mathcal{L}\left({\mathbb{F}}^4\right)}$ byT(z1,z2,z3,z4):=(0,3z1,2z2,-3z4)$$T(z_1,z_2,z_3,z_4):=(0,3z_1,2z_2,-3z_4)$$ThenT=a

3
	2
			-3

, T*=a

	3
		2
			-3

$$T=\left(\begin{array}{cccc}& & & \\ 3& & & \\ & 2& & \\ & & & -3\end{array}\right),T^{*}=\left(\begin{array}{cccc}& 3& & \\ & & 2& \\ & & & \\ & & & -3\end{array}\right)$$We can show that T*T=a

	3
		2
			-3

a

3
	2
			-3

=a

9
	4
		0
			9

$$T^{*}T=\left(\begin{array}{cccc}& 3& & \\ & & 2& \\ & & & \\ & & & -3\end{array}\right)\left(\begin{array}{cccc}& & & \\ 3& & & \\ & 2& & \\ & & & -3\end{array}\right)=\left(\begin{array}{cccc}9& & & \\ & 4& & \\ & & 0& \\ & & & 9\end{array}\right)$$So T*T${\displaystyle \sqrt{T^{*}T}}$ has three eigenvalues: 3, 2, 0. and the singular values of T${\displaystyle T}$ are 3, 3, 2, 0.

The next result shows that every operator on V${\displaystyle V}$ has a clean description in terms of its singular values and two orthonormal bases of V${\displaystyle V}$.

Theorem177Singular Value Decomposition

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ has singular values s1,⋯,sn${\displaystyle s_1,\cdots ,s_n}$. Then there exist orthonormal bases e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ and f1,⋯,fn${\displaystyle f_1,\cdots ,f_n}$ of V${\displaystyle V}$ such thatTv=s1<v,e1>f1+⋯+sn<v,en>fn$$Tv=s_1⟨v,e_1⟩f_1+\cdots +s_n⟨v,e_n⟩f_n$$for every v∈V${\displaystyle v\in V}$.Proof By the Spectral Theorem applied to T*T${\displaystyle \sqrt{T^{*}T}}$, there is an orthonormal basis e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ of V${\displaystyle V}$ such that T*Tej=sjej${\displaystyle \sqrt{T^{*}T}{e}_j=s_j{e}_j}$ for j=1,⋯,n${\displaystyle j=1,\cdots ,n}$. We havev=<v,e1>e1+⋯+<v,en>en$$v=⟨v,e_1⟩e_1+\cdots +⟨v,e_n⟩e_n$$for every v∈V${\displaystyle v\in V}$. Apply T*T ${\displaystyle \sqrt{T^{*}T}}$to both sides of this equation, gettingT*Tv=s1<v,e1>e1+⋯+sn<v,en>en$$\sqrt{T^{*}T}v=s_1⟨v,e_1⟩e_1+\cdots +s_n⟨v,e_n⟩e_n$$By the Polar Decomposition, there is an isometry S∈L(V)${\displaystyle S\in \mathcal{L}(V)}$ such that T=ST*T${\displaystyle T=S\sqrt{T^{*}T}}$. Let Sej=fj${\displaystyle S{e}_j=f_j}$, because S${\displaystyle S}$ is an isometry, f1,⋯,fn${\displaystyle f_1,\cdots ,f_n}$ is an orthonormal basis of V${\displaystyle V}$. The equation above now becomesTv=s1<v,e1>f1+⋯+sn<v,en>fn$$Tv=s_1⟨v,e_1⟩f_1+\cdots +s_n⟨v,e_n⟩f_n$$for every v∈V${\displaystyle v\in V}$.

TS∘T*Ta

s1
	s2
		s3

e1e2f1f2

Fig7:Singular Value Decomposition

The singular Value Decomposition allows us a rare opportunity to make good use of two different bases for the matrix of an operator. To do this, suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Let s1,⋯,sn${\displaystyle s_1,\cdots ,s_n}$ denote the singular values of T${\displaystyle T}$, and let e1,⋯,en${\displaystyle e_1,\cdots ,e_n}$ and f1,⋯,fn${\displaystyle f_1,\cdots ,f_n}$ be orthonormal bases of V${\displaystyle V}$ such that the Singular Value Decomposition holds. Because Tej=sjfj${\displaystyle T{e}_j=s_j{f}_j}$ for each j${\displaystyle j}$, we haveM(T,ej,fj)=a

s1		0
	⋱
		sn

$$\mathcal{M}(T,e_j,f_j)=\left(\begin{array}{ccc}{s}_1& & 0\\ & \ddots & \\ & & s_n\end{array}\right)$$ Chapter8. Operators on Complex Vector Spaces8.A Generalized Eigenvectors and Nilpotent OperatorsNull Spaces of Powers of an OperatorWe begin this chapter with a study of null spaces of powers of an operator.

Theorem178Sequence of increasing null spaces

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then{0}=null T0⊂null T1⊂⋯⊂null Tk⊂⋯$$\{0\}=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^0\subset \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^1\subset \cdots \subset \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^k\subset \cdots$$

Theorem179Equality in the sequence of null spaces

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Suppose m${\displaystyle m}$ is a nonnegative integer such that null Tm=null Tm+1${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^m=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+1}}$. Thennull Tm=null Tm+1=⋯=null Tm+k=⋯$$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^m=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+1}=\cdots =\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+k}=\cdots$$ProofSuppose v∈null Tm+k+1${\displaystyle v\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+k+1}}$, thenTm+1Tkv=Tm+k+1v=0$$T^{m+1}{T}^{k}v=T^{m+k+1}v=0$$HenceTkv∈null Tm+1=null Tm$$T^{k}v\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+1}=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^m$$ThereforeTmTkv=0$$T^m{T}^{k}v=0$$It means v∈null Tm+k${\displaystyle v\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+k}}$, null Tm+k+1⊂null Tm+k⊂null Tm+k+1${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+k+1}\subset \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+k}\subset \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{m+k+1}}$

Theorem180Null spaces stop growing

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Let n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. Thennull Tn=null Tn+1=null Tn+2=⋯$$\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{n+1}=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{n+2}=\cdots$$ProofWe need only prove that null Tn=null Tn+1${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{n+1}}$. Suppose this is not true. Then {0}=null T0⊊null T1⊊⋯⊊null Tn⊊null Tn+1$$\{0\}=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^0⊊\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^1⊊\cdots ⊊\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n⊊\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{n+1}$$At each of the strict inclusions in the chain above, the dimension increases by at least 1. Thus the dim null Tn+1⩾n+1${\displaystyle T^{n+1}⩾n+1}$

Unfortunately, it is not true that V=null T⊕range T${\displaystyle V=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}T\oplus \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}T}$ for each T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. However, the following result is a useful substitute.

Theorem181V${\displaystyle V}$ is the direct sum of null Tdim V${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{\mathrm{d}\mathrm{i}\mathrm{m}V}}$ and range Tdim V${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^{\mathrm{d}\mathrm{i}\mathrm{m}V}}$

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Let n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. ThenV=null Tn⊕range Tn.$$V=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\oplus \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n.$$Proof First we show that(null Tn)∩(range Tn)={0}$$\left(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\right)\cap \left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n\right)=\{0\}$$Suppose v∈(null Tn)∩(range Tn)${\displaystyle v\in \left(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\right)\cap \left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n\right)}$. Then Tnv=0${\displaystyle T^{n}v=0}$, and there exist u∈V${\displaystyle u\in V}$ such that v=Tnu${\displaystyle v=T^{n}u}$. Applying Tn${\displaystyle T^n}$ to both sides of the last equation shows that Tnv=T2nu${\displaystyle T^{n}v=T^{2n}u}$. Hence T2nu=0${\displaystyle T^{2n}u=0}$. u∈null T2n${\displaystyle u\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{2n}}$, so u∈null Tn${\displaystyle u\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n}$ too.

uTnnull Tnrange Tnv∈range Tn∩null Tn0u∈T2n=Tnv=0 And we know from the fundamental theorem of Linear Maps, dim null Tn=dim range Tn${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n}$. ThusV=null Tn⊕range Tn$$V=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\oplus \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n$$□

Generalized EigenvectorsSome operators do not have enough eigenvectors to lead to a good description. Thus in this subsection we introduce the concept of generalized eigenvectors. To understand why we need more than eigenvectors, let's examine the question of describing an operator by decomposing its domain into invariant subspaces. Fix T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. We seek to describe T${\displaystyle T}$ by finding a "nice" direct sum decompositionV=U1⊕⋯⊕Um$$V=U_1\oplus \cdots \oplus U_m$$where each Uj${\displaystyle U_j}$ is a subspace of V${\displaystyle V}$ invariant under T${\displaystyle T}$. The simplest possible nonzero invariant subspaces are 1-dimensional. A decomposition as above where each Uj${\displaystyle U_j}$ is 1-dimensional is possible if and only if V${\displaystyle V}$ has a basis consisting of eigenvectors of T${\displaystyle T}$. This happens if and only if V${\displaystyle V}$ has an eigenspace decomposition(20)V=E(𝜆1,T)⊕⋯⊕E(𝜆m,T)$$V=\mathbb{E}({𝜆}_1,T)\oplus \cdots \oplus \mathbb{E}({𝜆}_m,T)$$The Spectral Theorem in the previous chapter shows that if V${\displaystyle V}$ is an inner product space, then a decomposition of the form(20) holds for every normal operator if F=C${\displaystyle \mathbb{F}=\mathbb{C}}$ and for every self-adjoint operator if F=R${\displaystyle \mathbb{F}=\mathbb{R}}$ because operators of those types have enough eigenvectors to form a basis of V${\displaystyle V}$. Some operator does not have enough eigenvectors. Generalized eigenvectors and generalized eigenspaces, which we now introduce, will remedy this situation.

Definition182generalized eigenvector

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. and 𝜆${\displaystyle 𝜆}$ is an eigenvalue of T${\displaystyle T}$. A vector v∈V${\displaystyle v\in V}$ is called a generalized eigenvector of T${\displaystyle T}$ corresponding to 𝜆${\displaystyle 𝜆}$ if v≠0${\displaystyle v\ne 0}$ and (T-𝜆I)jv=0$$(T-𝜆\mathbb{I}{)}^{j}v=0$$for some positive integer j${\displaystyle j}$.

Definition183generalized eigenspaces G(𝜆,T)${\displaystyle G(𝜆,T)}$

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$. The generalized eigenspace of T${\displaystyle T}$ corresponding to 𝜆${\displaystyle 𝜆}$, denoted G(𝜆,T)${\displaystyle G(𝜆,T)}$, is defined to be the set of all generalized eigenvectors of T${\displaystyle T}$ corresponding to 𝜆${\displaystyle 𝜆}$, along with the 0${\displaystyle 0}$ vector.G(𝜆,T)=null (T-𝜆I)j=null (T-𝜆I)n${\displaystyle G(𝜆,T)=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(\mathrm{T}\mathrm{-}\mathrm{𝜆}\mathrm{I}{)}^{\mathrm{j}}\mathrm{=}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(T-𝜆\mathbb{I}{)}^n}$

Because every eigenvector of T${\displaystyle T}$ is a generalized eigenvector of T${\displaystyle T}$, each eigenspace is contained in the corresponding generalized eigenspace. In other words, E(𝜆,T)⊂G(𝜆,T)$$\mathbb{E}(𝜆,T)\subset G(𝜆,T)$$The next result implies that if T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$, then G(𝜆,T)${\displaystyle G(𝜆,T)}$ is a subspace of V${\displaystyle V}$

Theorem184Description of generalized eigenspaces

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Let n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. ThenV=null Tn⊕range Tn.$$V=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\oplus \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n.$$Proof First we show that(null Tn)∩(range Tn)={0}$$\left(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\right)\cap \left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n\right)=\{0\}$$Suppose v∈(null Tn)∩(range Tn)${\displaystyle v\in \left(\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\right)\cap \left(\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n\right)}$. Then Tnv=0${\displaystyle T^{n}v=0}$, and there exist u∈V${\displaystyle u\in V}$ such that v=Tnu${\displaystyle v=T^{n}u}$. Applying Tn${\displaystyle T^n}$ to both sides of the last equation shows that Tnv=T2nu${\displaystyle T^{n}v=T^{2n}u}$. Hence T2nu=0${\displaystyle T^{2n}u=0}$. u∈null T2n${\displaystyle u\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^{2n}}$, so u∈null Tn${\displaystyle u\in \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n}$ too. Thus v=Tnu=0${\displaystyle v=T^{n}u=0}$ And we know from the fundamental theorem of Linear Maps, dim null Tn=dim range Tn${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n}$. ThusV=null Tn⊕range Tn$$V=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{T}^n\oplus \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}{T}^n$$□

Example184.1

Define T∈L(C3)${\displaystyle T\in \mathcal{L}\left({\mathbb{C}}^3\right)}$ byT(z1,z2,z3)=(4z2,0,5z3)$$T(z_1,z_2,z_3)=(4z_2,0,5z_3)$$(a) Find all eigenvalues of T${\displaystyle T}$, the corresponding eigenspaces, and the corresponding generalized eigenspaces.(b) Show that C3${\displaystyle {\mathbb{C}}^3}$ is the direct sum of generalized eigenspaces corresponding to the distinct eigenvalues of T${\displaystyle T}$. (a) 𝜆1=0${\displaystyle {𝜆}_1=0}$, eigenvector is (z1,0,0)${\displaystyle (z_1,0,0)}$; 𝜆2=5${\displaystyle {𝜆}_2=5}$, eigenvector is (0,0,z3)${\displaystyle (0,0,z_3)}$. There must be other generalized eigenvector. We have T3(z1,z2,z3)=(0,0,125z3)${\displaystyle T^3(z_1,z_2,z_3)=(0,0,125z_3)}$ Thus v=(0,z2,0)${\displaystyle v=(0,z_2,0)}$ is also an eigenvector. Soa

G(0,T)={(z1,z2,0):z1,z2∈C}

G(5,T)={(0,0,z3):z3∈C}

$$\{\begin{array}{l}G(0,T)=\{(z_1,z_2,0):z_1,z_2\in \mathbb{C}\}\\ G(5,T)=\{(0,0,z_3):z_3\in \mathbb{C}\}\end{array}$$

One of our major goals in this chapter is to show that the result in part (b) of the example above holds in general for operators on finite-dimensional complex vector spaces.

Theorem185Linear independent generalized eigenvectors

Suppose 𝜆1,⋯,𝜆m${\displaystyle {𝜆}_1,\cdots ,{𝜆}_m}$ are distinct eigenvalues of T${\displaystyle T}$. v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ are corresponding generalized eigenvectors. Then v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ are linearly independent.Proof Suppose a1,⋯,am${\displaystyle a_1,\cdots ,a_m}$ are complex number such that(21)0=a1v1+⋯+amvm$$0=a_1{v}_1+\cdots +a_m{v}_m$$Let k${\displaystyle k}$ be the largest nonnegative integer such that (T-𝜆1I)kvk≠0${\displaystyle (T-{𝜆}_1\mathbb{I}{)}^k{v}_k\ne 0}$. Letw=(T-𝜆1I)kv1$$w=(T-{𝜆}_1\mathbb{I}{)}^k{v}_1$$Thus (T-𝜆1I)w=(T-𝜆1I)k+1w=0$$(T-{𝜆}_1\mathbb{I})w=(T-{𝜆}_1\mathbb{I}{)}^{k+1}w=0$$and hence Tw=𝜆1w${\displaystyle Tw={𝜆}_{1}w}$. Thus (T-𝜆I)w=(𝜆1-𝜆)w${\displaystyle (T-𝜆\mathbb{I})w=({𝜆}_1-𝜆)w}$ for every 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$ and hence(T-𝜆I)nw=(𝜆1-𝜆)nw$$(T-𝜆\mathbb{I}{)}^{n}w=({𝜆}_1-𝜆{)}^{n}w$$for every 𝜆∈F${\displaystyle 𝜆\in \mathbb{F}}$, where n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. Apply the operator (T-𝜆1I)k(T-𝜆2I)n⋯(T-𝜆m)n$$(T-{𝜆}_1\mathbb{I}{)}^k(T-{𝜆}_2\mathbb{I}{)}^n\cdots (T-{𝜆}_m{)}^n$$to both sides of (21). The equation above implies that a1=0${\displaystyle a_1=0}$. In a similar fashion, aj=0${\displaystyle a_j=0}$ for each j${\displaystyle j}$, which implies that v1,⋯,vm${\displaystyle v_1,\cdots ,v_m}$ is linearly independent.□

Nilpotent Operators

Definition186nilpotent

An operator is called nilpotent if some power of it equals 0${\displaystyle 0}$.

Example186.1

nilpotent

A. The operator N∈L(F4)${\displaystyle N\in \mathcal{L}\left({\mathbb{F}}^4\right)}$ defined byN(z1,z2,z3,z4)=(z3,z4,0,0)$$N(z_1,z_2,z_3,z_4)=(z_3,z_4,0,0)$$is nilpotentB. The operator of differentiation on P(R)${\displaystyle \mathcal{P}(\mathbb{R})}$ is nilpotent.

Theorem187Nilpotent operator raised to dimension of domain is 0${\displaystyle 0}$

Suppose N∈L(V)${\displaystyle N\in \mathcal{L}(V)}$ is nilpotent. Then NdimV=0${\displaystyle N^{\mathrm{d}\mathrm{i}\mathrm{m}V}=0}$ mapProofG(0,N)=V${\displaystyle G(0,N)=V}$, thus null NdimV=V${\displaystyle \mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}{N}^{\mathrm{d}\mathrm{i}\mathrm{m}V}=V}$, NdimV=0${\displaystyle N^{\mathrm{d}\mathrm{i}\mathrm{m}V}=0}$ map.

Theorem188Matrix of a nilpotent operator

Suppose N∈L(V)${\displaystyle N\in \mathcal{L}(V)}$ is nilpotent. Then there is a basis of V${\displaystyle V}$ with respect to which the matrix of N${\displaystyle N}$ has the forma

0		*
	⋱
0		0

$$\left(\begin{array}{ccc}0& & *\\ & \ddots & \\ 0& & 0\end{array}\right)$$ProofFirst choose a basis of null N${\displaystyle N}$. Then extend this to a basis of null N2${\displaystyle N^2}$. Then extend to a basis of null N3${\displaystyle N^3}$. Continue in this fashion, eventually getting a basis of V${\displaystyle V}$. Now let's think about the matrix of N${\displaystyle N}$ with respect to this basis. The first column, and perhaps additional columns at the beginning, consists of all 0${\displaystyle 0}$'s.a

?	?	?
?	?	?
?	?	?

a

x

0

0

=0⟶a

?	?	?
?	?	?
?	?	?

=a

0	?	?
0	?	?
0	?	?

$$\left(\begin{array}{ccc}?& ?& ?\\ ?& ?& ?\\ ?& ?& ?\end{array}\right)\left(\begin{array}{c}x\\ 0\\ 0\end{array}\right)=0⟶\left(\begin{array}{ccc}?& ?& ?\\ ?& ?& ?\\ ?& ?& ?\end{array}\right)=\left(\begin{array}{ccc}0& ?& ?\\ 0& ?& ?\\ 0& ?& ?\end{array}\right)$$Continue in this fashion:a

0	?	?
0	?	?
0	?	?

⟶a

0	?	?
0	0	?
0	0	?

⟶a

0	?	?
0	0	?
0	0	0

$$\left(\begin{array}{ccc}0& ?& ?\\ 0& ?& ?\\ 0& ?& ?\end{array}\right)⟶\left(\begin{array}{ccc}0& ?& ?\\ 0& 0& ?\\ 0& 0& ?\end{array}\right)⟶\left(\begin{array}{ccc}0& ?& ?\\ 0& 0& ?\\ 0& 0& 0\end{array}\right)$$

8.B Decomposition of an OperatorDescription of Operators on Complex Vector SpacesWe will see that every operator on a finite-dimensional complex vector space has enough generalized eigenvectors to provide a decomposition.

Theorem189The null space and range of p(T)${\displaystyle p(T)}$ are invariant under T${\displaystyle T}$

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and p∈P(F)${\displaystyle p\in \mathcal{P}(\mathbb{F})}$. Then null p(T)${\displaystyle p(T)}$ and range p(T)${\displaystyle p(T)}$ are invariant under T${\displaystyle T}$.ProofThe key is:p(T)(Tu)=Tp(T)u$$p(T)(Tu)=Tp(T)u$$

The following major result shows that every operator on a complex vector space can be thought of as composed of pieces, each of which is a nilpotent operator plus a scalar multiple of the identity.

Theorem190Description of operators on complex vector spaces

Suppose V${\displaystyle V}$ is a complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Let 𝜆1,⋯,𝜆m${\displaystyle {𝜆}_1,\cdots ,{𝜆}_m}$ be the distinct eigenvalues of T${\displaystyle T}$. Then(1) (22)V=G(𝜆1,T)⊕⋯⊕G(𝜆m,T)$$V=G({𝜆}_1,T)\oplus \cdots \oplus G({𝜆}_m,T)$$(2) each G(𝜆j,T)${\displaystyle G({𝜆}_j,T)}$ is invariant under T${\displaystyle T}$(3) each (T-𝜆jI)|G(𝜆j,T)${\displaystyle (T-{𝜆}_j\mathbb{I}){|}_{G({𝜆}_j,T)}}$ is nilpotentProof Let n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. Recall that G(𝜆j,T)=null(T-𝜆jI)n${\displaystyle G({𝜆}_j,T)=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(T-{𝜆}_j\mathbb{I}{)}^n}$ for each j${\displaystyle j}$. Thus G${\displaystyle G}$ is a polynomial's null spaces, and is invariant under T${\displaystyle T}$. (2) (3) holds. We will prove (1) by induction on n${\displaystyle n}$. To get started, note that the desired result holds if n=1${\displaystyle n=1}$. Thus we can assume that n>1${\displaystyle n>1}$ and that the desired result holds on all vector spaces of smaller dimension.Because V${\displaystyle V}$ is a complex vector space, T${\displaystyle T}$ has an eigenvalue; thus m⩾1${\displaystyle m⩾1}$. There exist a G(𝜆1,T)${\displaystyle G({𝜆}_1,T)}$. And we can decomposeV=G(𝜆1,T)⊕U$$V=G({𝜆}_1,T)\oplus U$$where U=range(T-𝜆1I)n${\displaystyle U=\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}(T-{𝜆}_1\mathbb{I}{)}^n}$. U${\displaystyle U}$ is a polynomial's range, so it is invariant under T${\displaystyle T}$. Because G(𝜆1,T)≠{0}${\displaystyle G({𝜆}_1,T)\ne \{0\}}$, we have dim U<n${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}U<n}$. Thus we can apply our induction hypothesis to T|U${\displaystyle T{|}_U}$. None of the generalized eigenvectors of T|U${\displaystyle T{|}_U}$ correspond to the eigenvalue 𝜆1${\displaystyle {𝜆}_1}$, because they would be in G(𝜆1,T)${\displaystyle G({𝜆}_1,T)}$. Thus each eigenvalue of T|U${\displaystyle T{|}_U}$ is in {𝜆2,⋯,𝜆m}${\displaystyle \{{𝜆}_2,\cdots ,{𝜆}_m\}}$. We need to show that G(𝜆k,T|U}=G(𝜆k,T)${\displaystyle G({𝜆}_k,T{|}_U\}=G({𝜆}_k,T)}$ for k=2,⋯,m${\displaystyle k=2,\cdots ,m}$. Thus fix k∈{2,⋯,m}${\displaystyle k\in \{2,\cdots ,m\}}$. The inclusion G(𝜆k, T|U)⊂G(𝜆k,T)${\displaystyle G({𝜆}_k,T{|}_U)\subset G({𝜆}_k,T)}$ is clear⋯${\displaystyle \cdots }$

G(𝜆,T)${\displaystyle G(𝜆,T)}$ cannot be decomposed into eigenvectors. For example:a

6	3	4
	6	2
		7

eigenvalue=6⏪⏪⏪⏪⏪⏫T-𝜆Ia

0	3	4
	0	2
		1

a

0	3	4
	0	2
		1

2=a

0	0	10
	0	2
		1

$$\begin{array}{c}\left(\begin{array}{ccc}6& 3& 4\\ & 6& 2\\ & & 7\end{array}\right)\underset{T-𝜆\mathbb{I}}{\overset{\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}=6}{\to }}\left(\begin{array}{ccc}0& 3& 4\\ & 0& 2\\ & & 1\end{array}\right)\\ {\left(\begin{array}{ccc}0& 3& 4\\ & 0& 2\\ & & 1\end{array}\right)}^2=\left(\begin{array}{ccc}0& 0& 10\\ & 0& 2\\ & & 1\end{array}\right)\end{array}$$Then the generalized eigenvectors are (1,0,0)${\displaystyle (1,0,0)}$ and (0,1,0)${\displaystyle (0,1,0)}$.

Theorem191A basis of generalized eigenvectors

Suppose V${\displaystyle V}$ is a complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then there is a basis of V${\displaystyle V}$ consisting of generalized eigenvectors of T${\displaystyle T}$.

Multiplicity of an EigenvalueIf V${\displaystyle V}$ is a complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$, then the decomposition of V${\displaystyle V}$ provided by (22) can be a powerful tool. The dimensions of the subspaces involved in this decomposition are sufficiently important to get a name.

Definition192multiplicity

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. The multiplicity of an eigenvalue 𝜆${\displaystyle 𝜆}$ of T${\displaystyle T}$ is defined to be the dimension of the corresponding generalized eigenspace G(𝜆,T)${\displaystyle G(𝜆,T)}$

Theorem193Sum of the multiplicities equals dimV${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V}$

Or we won't have equ(22).

More information

algebraic multiplicity of 𝜆=dim null(T-𝜆I)dimV=dim G(𝜆,T)${\displaystyle 𝜆=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(T-𝜆\mathbb{I}{)}^{\mathrm{d}\mathrm{i}\mathrm{m}V}=\mathrm{d}\mathrm{i}\mathrm{m}G(𝜆,T)}$geometric multiplicity of 𝜆=dim null(T-𝜆I)=dim E(𝜆,T)${\displaystyle 𝜆=\mathrm{d}\mathrm{i}\mathrm{m}\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(T-𝜆\mathbb{I})=\mathrm{d}\mathrm{i}\mathrm{m}\mathbb{E}(𝜆,T)}$

Block Diagonal MatricesTo interpret our results in matrix form, we make the following definition, generalizing the notion of a diagonal matrix

Definition194block diagonal matrix

square matrix of the forma

A1		0
	⋱
0		Am

$$\left(\begin{array}{ccc}{A}_1& & 0\\ & \ddots & \\ 0& & A_m\end{array}\right)$$where A1,⋯,Am${\displaystyle A_1,\cdots ,A_m}$ are square matrices lying along the diagonal and all the other entries of the matrix equal 0.

Theorem195Block diagonal matrix with upper-triangular blocks

We can choose a basis to block diagonalize an operator. Make every Aj${\displaystyle A_j}$ be a upper-trangular matrix.

Square RootsNot every operator on a complex vector space has a square root.

Theorem196Identity plus nilpotent has a square root

Suppose N∈L(V)${\displaystyle N\in \mathcal{L}(V)}$ is nilpotent. Then I+N${\displaystyle \mathbb{I}+N}$ has a square root.Proof Consider the Taylor series for the function 1+x${\displaystyle \sqrt{1+x}}$:1+a1x+a2x2+⋯$$1+a_{1}x+a_2{x}^2+\cdots$$Because N${\displaystyle N}$ is nilpotent, Nm=0${\displaystyle N^m=0}$ for some positive integer m${\displaystyle m}$. We guess that there is a square root of I+N${\displaystyle \mathbb{I}+N}$ of the formI+a1N+a2N2+⋯+am-1Nm-1$$\mathbb{I}+a_{1}N+a_2{N}^2+\cdots +a_{m-1}{N}^{m-1}$$Having this guess, we can compute(I+a1N+a2N2+⋯+am-1Nm-1)(I+a1N+a2N2+⋯+am-1Nm-1)$$(\mathbb{I}+a_{1}N+a_2{N}^2+\cdots +a_{m-1}{N}^{m-1})(\mathbb{I}+a_{1}N+a_2{N}^2+\cdots +a_{m-1}{N}^{m-1})$$and make the coefficient to be 1,1,0,0,0,⋯${\displaystyle 1,1,0,0,0,\cdots }$

Theorem197Over C${\displaystyle \mathbb{C}}$, invertible operators have square roots

Suppose V${\displaystyle V}$ is a complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ is invertible. Then T${\displaystyle T}$ has a square root.ProofInvertibility means all the eigenvalues are not zero. Thus G=T-𝜆I${\displaystyle G=T-𝜆\mathbb{I}}$ is a nilpotent, soT|G=𝜆(I+G

𝜆)$$T{|}_G=𝜆(\mathbb{I}+\frac{G}{𝜆})$$has a square root.

8.C Characteristic and Minimal PolynomialThe Cayley-Hamilton TheoremThe next definition associates a polynomial with each operator on V${\displaystyle V}$ if F=C${\displaystyle \mathbb{F}=\mathbb{C}}$.

Definition198characteristic polynomial

Suppose V${\displaystyle V}$ is a complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Let 𝜆1,⋯,𝜆m${\displaystyle {𝜆}_1,\cdots ,{𝜆}_m}$ denote the distinct eigenvalues of T${\displaystyle T}$, with multiplicities d1,⋯,dm${\displaystyle d_1,\cdots ,d_m}$. The polynomial (z-𝜆1)d1⋯(z-𝜆m)dm$$(z-{𝜆}_1{)}^{d_1}\cdots (z-{𝜆}_m{)}^{d_m}$$is called the characteristic polynomial of T${\displaystyle T}$.

Theorem199Cayley-Hamilton Theorem

Suppose V${\displaystyle V}$ is a complex vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Let q ${\displaystyle q}$denote the characteristic polynomial of T${\displaystyle T}$. Then q(T)=0${\displaystyle q(T)=0}$.Proof Every vector in V${\displaystyle V}$ is a sum of vectors in G(𝜆1,T),⋯G(𝜆m,T)${\displaystyle G({𝜆}_1,T),\cdots G({𝜆}_m,T)}$. Thus we only need to prove|p(T)|G(𝜆j,T)=0$$p(T){|}_{G({𝜆}_j,T)}=0$$Because p(T)=(T-𝜆1I)d1⋯${\displaystyle p(T)=(T-{𝜆}_1\mathbb{I}{)}^{d_1}\cdots }$, we can commute the term and let (T-𝜆jI)dj${\displaystyle (T-{𝜆}_j\mathbb{I}{)}^{d_j}}$ be the rightest. The definition of G${\displaystyle G}$ is:G=null(T-𝜆jI)dj$$G=\mathrm{n}\mathrm{u}\mathrm{l}\mathrm{l}(T-{𝜆}_j\mathbb{I}{)}^{d_j}$$ So the proof is clearly.

The Minimal Polynomial

Definition200monic polynomial

A monic polynomial is a polynomial whose highest-degree coefficient equals 1.

Theorem201Minimal polynomial

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then there is a unique monic polynomial p${\displaystyle p}$ of smallest degree such that p(T)=0${\displaystyle p(T)=0}$.ProofLet n=dim V${\displaystyle n=\mathrm{d}\mathrm{i}\mathrm{m}V}$. Then the listI, T, T2,⋯,Tn2$$\mathbb{I},T,T^2,\cdots ,T^{n^2}$$is linearly dependent, because dim L=n2${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathcal{L}=n^2}$. Let m${\displaystyle m}$ be the smallest positive integer such thatI, T, T2,⋯,Tm$$\mathbb{I},T,T^2,\cdots ,T^m$$is linearly dependent. soa0I+a1T+⋯+am-1Tm-1+Tm=0$$a_0\mathbb{I}+a_{1}T+\cdots +a_{m-1}{T}^{m-1}+T^m=0$$Define a monic polynomial p∈P(F)${\displaystyle p\in \mathcal{P}(\mathbb{F})}$ byp(z)=a0+a1z+⋯+am-1zm-1+zm$$p(z)=a_0+a_{1}z+\cdots +a_{m-1}{z}^{m-1}+z^m$$Then it means p(T)=0${\displaystyle p(T)=0}$.

Definition202minimal polynomial

A minimal polynomial of T${\displaystyle T}$ is the unique monic polynomial p${\displaystyle p}$ of smallest degree such thatp(T)=0$$p(T)=0$$

The proof of the last result shows that the degree of the minimal polynomial of each operator on V${\displaystyle V}$ is at most (dim V)2${\displaystyle (\mathrm{d}\mathrm{i}\mathrm{m}V{)}^2}$. The Cayley-Hamilton Theorem tells us that if V ${\displaystyle V}$is a complex vector space, then the minimal polynomial of each operator on V${\displaystyle V}$ has degree at most dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V}$. This remarkable improvement also holds on real vector spaces, as we will see in the next chapter.

Theorem203q(T)=0${\displaystyle q(T)=0}$ implies q${\displaystyle q}$ is a multiple of the minimal polynomial

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ and q∈P(F)${\displaystyle q\in \mathcal{P}(\mathbb{F})}$. Then q(T)=0${\displaystyle q(T)=0}$ if and only if q${\displaystyle q}$ is a polynomial multiple of the minimal polynomial of T${\displaystyle T}$.Proof The "if" direction is easy to prove. The other direction, use the Division Algorithm for Polynomials, there exist polynomials s,r∈P(F)${\displaystyle s,r\in \mathcal{P}(\mathbb{F})}$ such thatq=ps+r$$q=ps+r$$and deg r<deg p${\displaystyle \mathrm{d}\mathrm{e}\mathrm{g}r<\mathrm{d}\mathrm{e}\mathrm{g}p}$. We can prove that r=0${\displaystyle r=0}$

Theorem204Characteristic polynomial is a multiple of minimal polynomial

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$, F=C${\displaystyle \mathbb{F}=\mathbb{C}}$. Then the characteristic polynomial of T${\displaystyle T}$ is a polynomial multiple of the minimal polynomial of T${\displaystyle T}$.

Theorem205Eigenvalues are the zeros of the minimal polynomial

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then the zeros of the minimal polynomial of T${\displaystyle T}$ are precisely the eigenvalues of T${\displaystyle T}$.

8.D Jordan FormWe know that if V${\displaystyle V}$ is a complex vector space, then for every T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$ there is a basis of V${\displaystyle V}$ with respect to which T${\displaystyle T}$ has a nice upper-triangular matrix. In this section we will see that we can do even better -- there is a basis of V${\displaystyle V}$ with respect to which the matrix of T${\displaystyle T}$ contains 0's everywhere except possibly on the diagonal and the line directly above the diagonal.

Theorem206Basis corresponding to a nilpotent operator

Suppose N∈L(V)${\displaystyle N\in \mathcal{L}(V)}$ is a nilpotent. Then there exist vectors v1,⋯,vn∈V${\displaystyle v_1,\cdots ,v_n\in V}$ and nonnegative integers m1,⋯,mn${\displaystyle m_1,\cdots ,m_n}$ such that(1) (23)Nm1v1,⋯,Nv1,v1⋯,Nmnvn,⋯,Nvn,vn is a basis of V$$N^{m_1}{v}_1,\cdots ,N{v}_1,v_1\cdots ,N^{m_n}{v}_n,\cdots ,N{v}_n,v_n\mathrm{i}\mathrm{s}\mathrm{a}\mathrm{b}\mathrm{a}\mathrm{s}\mathrm{i}\mathrm{s}\mathrm{o}\mathrm{f}V$$(2) (24)Nm+1v1=⋯=Nmn+1vn=0$$N^{m+1}{v}_1=\cdots =N^{m_n+1}{v}_n=0$$ProofUse the induction on dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V}$. When dim V=1${\displaystyle V=1}$, N${\displaystyle N}$ is must a 0${\displaystyle 0}$ map. So let m=0${\displaystyle m=0}$, v1${\displaystyle v_1}$is a basis of V${\displaystyle V}$. Now for a N${\displaystyle N}$ and V${\displaystyle V}$, Suppose the theorem holds when dim V'<dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}V\text{'}<\mathrm{d}\mathrm{i}\mathrm{m}V}$. Thus It should holds on the range of N${\displaystyle N}$ because dim range N<dim V${\displaystyle \mathrm{d}\mathrm{i}\mathrm{m}\mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}N<\mathrm{d}\mathrm{i}\mathrm{m}V}$. Thus(25)Nm1v1,⋯,v1,⋯,Nmnvn,⋯,vn$$N^{m_1}{v}_1,\cdots ,v_1,\cdots ,N^{m_n}{v}_n,\cdots ,v_n$$is a basis of range N${\displaystyle \mathrm{r}\mathrm{a}\mathrm{n}\mathrm{g}\mathrm{e}N}$. for each vj${\displaystyle v_j}$, there exist uj${\displaystyle u_j}$ such that vj=Nuj${\displaystyle v_j=N{u}_j}$. We will prove that (26)Nm1+1u1,⋯,u1,⋯,Nmn+1un,⋯,un$$N^{m_1+1}{u}_1,\cdots ,u_1,\cdots ,N^{m_n+1}{u}_n,\cdots ,u_n$$can extend to a basis by two steps. First, Nm1+1u1,⋯,u1,⋯,Nmn+1un,⋯,un${\displaystyle N^{m_1+1}{u}_1,\cdots ,u_1,\cdots ,N^{m_n+1}{u}_n,\cdots ,u_n}$ is linearly independent. If there were(27)a1u1+⋯am1+1Nm1+1u1+⋯=0$$a_1{u}_1+\cdots a_{m_1+1}{N}^{m_1+1}{u}_1+\cdots =0$$Then operator N${\displaystyle N}$ on the left, most of the vector become the vector in (25), others become 0${\displaystyle 0}$ because of (24). (25) is linearly independent, thus their coefficients must be 0. Only those who became 0${\displaystyle 0}$ can have nonnegative coefficients. However, their origin is actually Nm1v1,⋯,Nmnvn${\displaystyle N^{m_1}{v}_1,\cdots ,N^{m_n}{v}_n}$, which is also linearly independent. So (27) requires their coefficients to be 0. Second, extend (26) to be a basis(28)Nm1+1u1,⋯,u1,⋯,Nmn+1un,⋯,un,w1,⋯,wp$$N^{m_1+1}{u}_1,\cdots ,u_1,\cdots ,N^{m_n+1}{u}_n,\cdots ,u_n,w_1,\cdots ,w_p$$of V${\displaystyle V}$. Each Nwj${\displaystyle N{w}_j}$ is in range N${\displaystyle N}$ and hence is in the span of (25) . Each vector in the list (25) equals N${\displaystyle N}$ applied to some vector in the list (26). Thus there exists xj${\displaystyle x_j}$ in the span of (26) such thatNwj=Nxj$$N{w}_j=N{x}_j$$Now letun+j=wj-xj$$u_{n+j}=w_j-x_j$$Then Nun+j=0${\displaystyle N{u}_{n+j}=0}$. Furthermore,Nm1+1u1,⋯,Nu1,u1,⋯Nmn+1un,⋯,Nun,un,un+1,⋯,un+p$$N^{m_1+1}{u}_1,\cdots ,N{u}_1,u_1,\cdots N^{m_n+1}{u}_n,\cdots ,N{u}_n,u_n,u_{n+1},\cdots ,u_{n+p}$$spans V${\displaystyle V}$ because it span contains each xj${\displaystyle x_j}$ and each un+j${\displaystyle u_{n+j}}$ and hence each wj${\displaystyle w_j}$. Thus the spanning list above is a basis of V${\displaystyle V}$ because it has the same length as the basis (28). This basis has the required form, where mn+j=0${\displaystyle m_{n+j}=0}$ for j⩾1${\displaystyle j⩾1}$.

Definition207Jordan basis

Suppose T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. A basis of V${\displaystyle V}$ is called a Jordan basis for T${\displaystyle T}$ if with respect to this basis T${\displaystyle T}$ has a block diagonal matrixa

A1
	⋱
		Ap

, where Aj=a

𝜆j	1		0
	⋱	⋱
		⋱	1
0			𝜆j

$$\left(\begin{array}{ccc}{A}_1& & \\ & \ddots & \\ & & A_p\end{array}\right),\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}{A}_j=\left(\begin{array}{cccc}{𝜆}_j& 1& & 0\\ & \ddots & \ddots & \\ & & \ddots & 1\\ 0& & & {𝜆}_j\end{array}\right)$$

Theorem208Jordan Form

Suppose V${\displaystyle V}$ is a complex vector space. If T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$, then there is a basis of V${\displaystyle V}$ that is a Jordan basis for T${\displaystyle T}$. Proof(23) tells us, a nilpotent operator can gives a basis of V${\displaystyle V}$ with respect to which N${\displaystyle N}$ has a block diagonal matrix, where each matrix on the diagonal has the form(29)a

a 0 1 0 ⋱ ⋱ ⋱ 1 0 0
	a 0 1 0 ⋱ ⋱ ⋱ 1 0 0
		a 0 1 0 ⋱ ⋱ ⋱ 1 0 0

$$\left(\begin{array}{ccc}\left(\begin{array}{cccc}0& 1& & 0\\ & \ddots & \ddots & \\ & & \ddots & 1\\ 0& & & 0\end{array}\right)& & \\ & \left(\begin{array}{cccc}0& 1& & 0\\ & \ddots & \ddots & \\ & & \ddots & 1\\ 0& & & 0\end{array}\right)& \\ & & \left(\begin{array}{cccc}0& 1& & 0\\ & \ddots & \ddots & \\ & & \ddots & 1\\ 0& & & 0\end{array}\right)\end{array}\right)$$Because (T-𝜆iI)|G(𝜆j,T)${\displaystyle (T-{𝜆}_i\mathbb{I}){|}_{G({𝜆}_j,T)}}$ is nilpotent. Thus (T-𝜆iI)|G(𝜆j,T)${\displaystyle (T-{𝜆}_i\mathbb{I}){|}_{G({𝜆}_j,T)}}$ can be represented as(29). So T|G(𝜆j,T)${\displaystyle T{|}_{G({𝜆}_j,T)}}$ can be written asa

a 𝜆i 1 0 ⋱ ⋱ ⋱ 1 0 𝜆i
	a 𝜆i 1 0 ⋱ ⋱ ⋱ 1 0 𝜆i
		a 𝜆i 1 0 ⋱ ⋱ ⋱ 1 0 𝜆i

$$\left(\begin{array}{ccc}\left(\begin{array}{cccc}{𝜆}_i& 1& & 0\\ & \ddots & \ddots & \\ & & \ddots & 1\\ 0& & & {𝜆}_i\end{array}\right)& & \\ & \left(\begin{array}{cccc}{𝜆}_i& 1& & 0\\ & \ddots & \ddots & \\ & & \ddots & 1\\ 0& & & {𝜆}_i\end{array}\right)& \\ & & \left(\begin{array}{cccc}{𝜆}_i& 1& & 0\\ & \ddots & \ddots & \\ & & \ddots & 1\\ 0& & & {𝜆}_i\end{array}\right)\end{array}\right)$$And we have the generalized eigenspace decomposition (22):V=G(𝜆1,T)⊕⋯⊕G(𝜆m,T)$$V=G({𝜆}_1,T)\oplus \cdots \oplus G({𝜆}_m,T)$$Thus we can represent T|V${\displaystyle T{|}_V}$ as the block diagonal matrix of T|G(𝜆j,T)${\displaystyle T{|}_{G({𝜆}_j,T)}}$. Each block has several blocks.

Chapter9. Operators on Real Vector Spaces9.A Complexification of a Vector SpaceAs we will soon see, a real vector space V${\displaystyle V}$ can be embedded, in a natural way, in a complex vector space called the complexification of V${\displaystyle V}$

Definition209complexification of V${\displaystyle V}$, VC${\displaystyle V_{\mathbb{C}}}$

Suppose V${\displaystyle V}$ is a real vector space.★ The complexification of V${\displaystyle V}$, denoted VC${\displaystyle V_{\mathbb{C}}}$, equals V×V${\displaystyle V\times V}$. An element of VC${\displaystyle V_C}$ is an ordered pair (u,v)${\displaystyle (u,v)}$, where u,v∈V${\displaystyle u,v\in V}$, but we will write this as u+ⅈv${\displaystyle u+iv}$.★ Addition on VC${\displaystyle V_C}$ is defined by(u1+ⅈv1)+(u2+ⅈv2)=(u1+u2)+ⅈ(v1+v2)$$(u_1+i{v}_1)+(u_2+i{v}_2)=(u_1+u_2)+i(v_1+v_2)$$★ Complex scalar multiplication on VC${\displaystyle V_C}$ is defined by(a+bⅈ)(u+vⅈ)=(au-bv)+ⅈ(av+bu)$$(a+bi)(u+vi)=(au-bv)+i(av+bu)$$★ for a,b∈R${\displaystyle a,b\in \mathbb{R}}$ and u,v∈V${\displaystyle u,v\in V}$.

We think of V${\displaystyle V}$ as a subset of VC${\displaystyle V_{\mathbb{C}}}$ by identifying u∈V${\displaystyle u\in V}$ with u+ⅈ0${\displaystyle u+i0}$. The construction of VC${\displaystyle V_{\mathbb{C}}}$ from V ${\displaystyle V}$can then by thought of as generalizing the construction of Cn${\displaystyle {\mathbb{C}}^n}$ from Rn${\displaystyle {\mathbb{R}}^n}$

Theorem210basis of V${\displaystyle V}$ is basis of VC${\displaystyle V_{\mathbb{C}}}$

Suppose V${\displaystyle V}$ is a real vector space★ If v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of V${\displaystyle V}$(as a real vector space, the coefficients are real), then v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ is a basis of VC${\displaystyle V_{\mathbb{C}}}$ (as a complex vector space, the coefficients are complex)★ their dimensions equal.

Complexification of an Operator

Definition211complexification of T${\displaystyle T}$, TC${\displaystyle T_{\mathbb{C}}}$

Suppose V${\displaystyle V}$ is a real vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. The complexification of T${\displaystyle T}$, denoted TC${\displaystyle T_{\mathbb{C}}}$, is the operator TC∈L(VC)${\displaystyle T_{\mathbb{C}}\in \mathcal{L}(V_{\mathbb{C}})}$ defined byTC(u+ⅈv)=Tu+ⅈTv$$T_{\mathbb{C}}(u+iv)=Tu+iTv$$for u,v∈V${\displaystyle u,v\in V}$.

Theorem212Matrix of TC ${\displaystyle T_{\mathbb{C}}}$equals matrix of T${\displaystyle T}$

Suppose V${\displaystyle V}$ is a real vector space with basis v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then M(T)=M(TC)${\displaystyle \mathcal{M}(T)=\mathcal{M}(T_{\mathbb{C}})}$, where both matrices are with respect to the same basis

We know that every operator on a nonzero finite-dimensional complex vector space has an eigenvalue and thus has a 1-dimensional invariant subspace. But an operator on a nonzero finite-dimensional real vector space may have no eigenvalues and thus no 1-dimensional invariant subspaces. However, we now show that an invariant subspace of dimension 1 or 2 always exists.

Theorem213Every operator has an invariant subspace of dimension 1 or 2.

Suppose V${\displaystyle V}$ is a real vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. The complexification TC${\displaystyle T_{\mathbb{C}}}$ has an eigenvalue a+bⅈ${\displaystyle a+bi}$, where a,b∈R${\displaystyle a,b\in \mathbb{R}}$. Thus there exist u,v∈V${\displaystyle u,v\in V}$, not both 0${\displaystyle 0}$, such that TC(u+ⅈv)=(a+bⅈ)(u+ⅈv)${\displaystyle T_{\mathbb{C}}(u+iv)=(a+bi)(u+iv)}$. Using the definition of TC${\displaystyle T_{\mathbb{C}}}$, the last equation can be rewritten asTu+ⅈTv=(au-bv)+(av+bu)ⅈ$$Tu+iTv=(au-bv)+(av+bu)i$$ThusTu=au-bv and Tv=av+bu$$Tu=au-bv\mathrm{a}\mathrm{n}\mathrm{d}Tv=av+bu$$Then U=span(u,v)${\displaystyle U=\mathrm{s}\mathrm{p}\mathrm{a}\mathrm{n}(u,v)}$ is invariant.

The Minimal Polynomial of the ComplexificationSuppose V${\displaystyle V}$ is a real vector space and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then the minimal polynomial of TC${\displaystyle T_{\mathbb{C}}}$ equals the minimal polynomial of T${\displaystyle T}$. Eigenvalues of the ComplexificationAn eigenvalue of TC${\displaystyle T_{\mathbb{C}}}$ is real if and only if it is also an eigenvalue of T${\displaystyle T}$.

Theorem214TC-𝜆I${\displaystyle T_{\mathbb{C}}-𝜆\mathbb{I}}$ and TC-⏨𝜆I${\displaystyle T_{\mathbb{C}}-\overline{𝜆}\mathbb{I}}$

(TC-𝜆I)j(u+ⅈv)=0${\displaystyle (T_{\mathbb{C}}-𝜆\mathbb{I}{)}^j(u+iv)=0}$ if and only if (TC-⏨𝜆I)j(u+ⅈv)=0${\displaystyle (T_{\mathbb{C}}-\overline{𝜆}\mathbb{I}{)}^j(u+iv)=0}$ As a consequence, the nonreal eigenvalues of TC${\displaystyle T_{\mathbb{C}}}$ comes in pair.

Theorem215Multiplicity of 𝜆${\displaystyle 𝜆}$ equals multiplicity of ⏨𝜆${\displaystyle \overline{𝜆}}$

Suppose V${\displaystyle V}$ is a real vector space with basis v1,⋯,vn${\displaystyle v_1,\cdots ,v_n}$ and T∈L(V)${\displaystyle T\in \mathcal{L}(V)}$. Then M(T)=M(TC)${\displaystyle \mathcal{M}(T)=\mathcal{M}(T_{\mathbb{C}})}$, where both matrices are with respect to the same basis

Chapter10. Trace and Determinant10.A TraceChange of Basis★ identity matrix I${\displaystyle \mathbb{I}}$★ invertible, inverse, A-1${\displaystyle A^{-1}}$★ The matrix of the product of linear mapsM(ST,(u1,⋯,un),(w1,⋯,wn))=M(S,(v1,⋯,vn),(w1,⋯,wn))M(T,(u1,⋯,un),(v1,⋯,vn))$$\mathcal{M}(ST,(u_1,\cdots ,u_n),(w_1,\cdots ,w_n))=\mathcal{M}(S,(v_1,\cdots ,v_n),(w_1,\cdots ,w_n))\mathcal{M}(T,(u_1,\cdots ,u_n),(v_1,\cdots ,v_n))$$★ M(I,(u1,⋯,un),(v1,⋯,vn))${\displaystyle \mathcal{M}(\mathbb{I},(u_1,\cdots ,u_n),(v_1,\cdots ,v_n))}$'s inverse is M(I,(v1,⋯,vn),(u1,⋯,un))${\displaystyle \mathcal{M}(\mathbb{I},(v_1,\cdots ,v_n),(u_1,\cdots ,u_n))}$★ Change of basis formula, let A=M(I,(u1,⋯,un),(v1,⋯,vn))${\displaystyle A=\mathcal{M}(\mathbb{I},(u_1,\cdots ,u_n),(v_1,\cdots ,v_n))}$M(T,(u1,⋯,un))=A-1M(T,(v1,⋯,vn))A$$\mathcal{M}(T,(u_1,\cdots ,u_n))=A^{-1}\mathcal{M}(T,(v_1,\cdots ,v_n))A$$ Trace: A Connection Between Operators and Matrices★ The trace of T${\displaystyle T}$ is the sum of the eigenvalues of T${\displaystyle T}$ (if F=C${\displaystyle \mathbb{F}=\mathbb{C}}$)or TC${\displaystyle T_{\mathbb{C}}}$(if F=R)${\displaystyle \mathbb{F}\mathbb{=}\mathbb{R})}$★ Trace T${\displaystyle T}$ equals the negative of the coefficient of zn-1${\displaystyle z^{n-1}}$ in the characteristic polynomial of T${\displaystyle T}$.★ The trace of a square matrix is the sum of the diagonal entries of it.★ trace(AB)=${\displaystyle (AB)=}$trace(BA)${\displaystyle (BA)}$★ Trace of matrix of operator does not depend on basis: T1=A-1T2A${\displaystyle T_1=A^{-1}{T}_{2}A}$, traceT1=${\displaystyle T_1=}$traceT2${\displaystyle T_2}$★ Trace of an operator equals trace of its matrix★ Trace is additive10.B DeterminantDeterminant of an operator★ It's the product of the eigenvalue of T${\displaystyle T}$ or TC${\displaystyle T_{\mathbb{C}}}$, with each eigenvalue repeated according to its multiplicity★ det T${\displaystyle T}$ equals (-1)n${\displaystyle (-1{)}^n}$ times the constant term of the characteristic polynomial of T${\displaystyle T}$.★ Invertible is equivalent to nonzero determinant★ Characteristic polynomial of T${\displaystyle T}$ equals det(zI-T)${\displaystyle (z\mathbb{I}-T)}$Determinant of a matrix★ detA= ∑(m1,⋯,mn)∈perm n(sign(m1,⋯,mn))Am1,1⋯Amn,n${\displaystyle A=\sum _{(m_1,\cdots ,m_n)\in \mathrm{p}\mathrm{e}\mathrm{r}\mathrm{m}n}^{}(\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}(m_1,\cdots ,m_n))A_{m_1,1}\cdots A_{m_n,n}}$★ determinant is multiplicative★ Determinant of an operator equals determinant of its matrix

Acknowledgment

All the typesetting is completed using Math. Math really kicks ass!