Section 2 ran Gram–Schmidt on with , and stopped at degree only because the space
did (Example (Gram–Schmidt on )).
Nothing in the process cared. Feed it
and it returns an infinite orthogonal sequence, one
polynomial of each degree. Three things make such
sequences worth a section. Each member is computable
from the two before it. Each member of degree has distinct real roots
inside the interval. And those roots are the best
possible points at which to sample a function to
estimate its integral. The first fact rests on one small
identity; the other two rest on the first.
Throughout this section the field is .
Two ways
to make an inner product on polynomials
Both constructions we need appeared in Section 1.
From a weight. Fix an interval with
endpoints and a weight
: a function
continuous and strictly positive on the
open interval , with finite for every
. Define
Finiteness of the moments makes this a real
number for all , and bilinearity and symmetry come from
linearity of the integral. For positivity, let . Then is continuous and
on , and off the finitely
many roots of (Corollary 6.1.4), so
. That last step is the one calculus fact this
section leans on: a continuous non-negative function on
an interval whose integral is vanishes identically.
As in Definition 11.1.1
it follows from the local fact that a continuous
function non-zero at a point stays bounded away from
on a small
interval around it, which then contributes a positive
amount to the integral.
From nodes. Fix distinct
in and put
. Section 1 checked
that this is an inner product on ,
positivity coming from the root bound. It is not one on
: the
polynomial is non-zero and pairs with itself to
give . So in the
node case everything below stops at degree .
The two constructions share one property, and it is
the engine of the section.
Proposition
11.10.1 (Multiplication by x is
symmetric)
Let be
an inner product on coming from a
weight, or the node inner product on . Then
for all polynomials in the space (in the
node case, for all of degree at most , so that and still lie in it).
Proof. In the weight case both
sides equal , and in the node case
both sides equal .
The proof is one line, but the statement is not a
triviality: multiplication by moves from one slot to
the other for free. That is exactly the identity defining a
self-adjoint operator (Definition 11.6.2),
and it is the reason the recurrence below has three
terms instead of .
Warning. Multiplication by
is not an
operator on . It raises degree, so it maps that
space out of itself; on it is an operator,
but there the space is infinite-dimensional, and Definition 11.6.2
was stated for a finite-dimensional one. “Self-adjoint”
is here a name for Proposition 11.10.1,
not a literal application of the definition. It is the
identity, not the name, that the proofs use.
Orthogonal polynomial sequences
Applied to the independent list ,
Gram–Schmidt never stalls. What comes out is one
polynomial of each degree, pairwise orthogonal, and that
combination deserves a name.
An orthogonal polynomial sequence is one
polynomial of every degree, no two of them pairing to
anything but zero.
Definition
11.10.1 (Orthogonal Polynomial Sequence)
Let be
an inner product on , or on . An
orthogonal polynomial sequence for it
is a sequence of polynomials (a finite list in the
second case) such that
- (OP1) for
every index ;
- (OP2)
whenever .
The sequence is monic if every has leading
coefficient .
In words. (OP1) says one polynomial of each degree:
no gaps, no repeats, no accidental drop in degree. (OP2)
says the members are pairwise orthogonal. Together they
say more than either alone, because by (OP1) the list
is a basis of , so
(OP2) upgrades to and that is the form in which the condition
always gets used.
Two things need checking: that such a sequence
exists, and that fixing the scale pins it down.
Proposition
11.10.2 (Existence and uniqueness of the monic
sequence)
Let be
an inner product on . For each there is
exactly one monic of degree with for
every . These form the unique monic
orthogonal polynomial sequence, and is an
orthogonal basis of for every
. The same holds
on
for .
Idea. Existence is Gram–Schmidt,
once we know the -th output really has
degree ; it does,
because it lies in but not in
the span of its predecessors, which is .
Uniqueness is the standard move: two candidates differ
by a polynomial of lower degree that is orthogonal to
everything of lower degree, hence to itself.
Proof. Existence. The list
is independent, so Theorem 11.2.2
applies and returns an orthonormal list with
for
every . Fix
. Then ,
and , since an
orthonormal list is independent (Theorem 11.1.4).
Hence
exactly. Let be its leading coefficient and put , monic of
degree . For we get by
orthogonality of the list.
Uniqueness. Suppose and are both monic of
degree and
orthogonal to every element of . Their
leading terms cancel, so . Therefore , and positive definiteness gives .
Finally, is an orthogonal list of non-zero vectors,
hence independent (Theorem 11.1.4),
with members, hence a basis (Theorem 1.6.4).
Examples. For the first three monic members are , , (Example (Gram–Schmidt on ));
these are the monic Legendre
polynomials. For the node inner product with
nodes on
we
have , , , , , so , , — and there the sequence ends, since
holds nothing of degree . The degenerate member
is always ,
the only monic polynomial of degree , with an empty
condition on it.
A non-example by minimal change.
Replace the Legendre by . Still monic of degree , so (OP1) survives; but
, so (OP2) fails.
Adding a multiple of a lower member is exactly the
freedom orthogonality removes.
Why monic? Rescaling each by a non-zero gives another
orthogonal polynomial sequence, so a normalization has
to be chosen, and traditions differ: the classical
Legendre polynomials are fixed by , and
Gram–Schmidt as run in Section 2 fixes . Monic is
the convenient choice here, because it is the one for
which the recurrence below needs no extra factor.
Warning. The sequence
belongs to the inner product, not to the
polynomials. “The orthogonal polynomials” is
meaningless without a weight and an interval. Exercise
B2 of Section 2 (Exercise (B2: Gram–Schmidt on polynomials))
ran the same process with instead of and got
, , : same
degrees, different polynomials.
Quick check. For , how many polynomials of degree with are
orthogonal to ? Write them down.
Solution. Exactly two. If has degree and ,
let
be its leading coefficient; then is monic of
degree and still
orthogonal to , so by Proposition 11.10.2.
Thus , and leaves . Since Example (Gram–Schmidt on )
computed , the two answers
are . Fixing the scale by the norm leaves a sign;
fixing it by “monic” leaves nothing.
The
three-term recurrence
Computing by
Gram–Schmidt costs one inner product for every earlier
member, so reaching degree costs about integrals. Almost
all of them return . The next theorem says
which ones do not, and reduces the cost to two integrals
per step.
Theorem
11.10.3 (Three-Term Recurrence)
Let be the
monic orthogonal polynomial sequence of Proposition 11.10.2,
for an inner product satisfying Proposition 11.10.1.
For a weight on an interval this covers every ; for the node inner
product on nodes,
where Proposition 11.10.1
holds only up to degree , read the statement for . Put and, for , Then , , and Moreover for every .
Idea. The polynomial is monic of degree
, so it is
plus
something of degree at most , and that something
expands in the orthogonal basis with
coefficients
(Corollary 11.3.4).
The point is that almost every coefficient is : move the across with Proposition 11.10.1,
and the coefficient becomes , which
vanishes as soon as is
still below . Only
and can
survive.
Proof. is the unique
monic polynomial of degree , and is the
unique monic polynomial of degree with ,
since exactly for
.
Now fix .
By Proposition 11.10.2,
is an orthogonal basis of , and
lies in that
space. By Corollary 11.3.4
applied inside , Both
and are
monic of degree , and every other term of the sum has degree
at most ;
comparing coefficients of gives .
Let .
By Proposition 11.10.1,
, and , so and the pairing is by Proposition 11.10.2.
Hence for
every ,
and which rearranges to .
It remains to identify the two coefficients. By
definition . For
the other,
is monic of degree , so has degree
at most and
therefore pairs to with . Hence, using Proposition 11.10.1
again, so . Finally is a quotient of
by
,
both strictly positive because and are non-zero, so
.
Two inner products per step generate the whole
sequence. The positivity of is not decoration
either: it is the sign that drives the interlacing
argument of Exercise (C1: Interlacing roots).
Example
(The monic Legendre polynomials, continued)
For , Example (Gram–Schmidt on )
gave , , ,
with , and . Use Theorem 11.10.3 to
obtain , and
check the recurrence reproduces .
Solution. The weight is even on , so every by the remark
above. Then The recurrence at gives , which is what Gram–Schmidt
produced. At ,
Check. because the integrands are odd, and So , as it should be. Two integrals
produced a polynomial that Gram–Schmidt would have
charged three for, and the saving grows with the
degree.
Where the
roots are
The roots of are , and those of
are and . All five lie inside , where the weight
lives. No coincidence. The argument first needs one fact
about real polynomials, in which
is the multiplicity of as a root of (Definition 6.4.1).
Lemma
11.10.4 (Even multiplicities mean no sign
change)
Let be an open interval and let be non-zero.
If
is even for every , then throughout , or throughout .
Proof. By Theorem 6.4.3 over
, write , where are the
distinct real roots of , , and has no real root.
Separate the factors according to whether : Each
with is
even by hypothesis, so the second factor is the square
of a real polynomial and is everywhere. The
polynomial has no
root in : its
roots are the
outside , and
has none. Being
continuous and nowhere zero on the interval , has constant sign there
by the intermediate value theorem. Hence has that same sign
throughout ,
weakly, with equality only at the inside .
Theorem
11.10.5 (Roots of an Orthogonal
Polynomial)
Let be a
weight on ,
let , and let be the monic
orthogonal polynomial sequence. For every , the polynomial
has distinct
real roots, and every one of them lies in the
open interval .
Idea. Suppose changed sign at only
a few points of , with . Build the polynomial . It is cheap — degree at most — so is orthogonal to it.
But was designed
to change sign at exactly the points where does, so the product
changes sign
nowhere, and a function of one sign that is not
identically zero cannot integrate to zero against a
positive weight. The contradiction forces , and distinct roots is all a
polynomial of degree can hold.
Proof. Fix . Let
be the roots of
in whose
multiplicity is odd (possibly ), and set , with when .
Suppose, for contradiction, that . Then ,
so by Proposition 11.10.2.
Put , a
non-zero polynomial (Theorem 0.8.3). Let
be a
root of . By Theorem 6.4.2, . If for some , then and
is odd by the choice of the , so the sum is even.
If is none of the
, then , so and
is even, again by the choice of the . Either way
is even.
By Lemma 11.10.4,
throughout
or throughout ; replacing by if necessary, which
changes neither nor
, we may
assume
there. Then is
continuous and on , and it is off the finitely many roots of , because . By the calculus
fact recalled when the inner product was set up — a
continuous non-negative function with integral vanishes identically —
an integrand of that shape cannot integrate to . Therefore contradicting .
Hence .
Each is a root
of , and is non-zero of degree
, so by Corollary 6.1.4; thus
and has distinct roots , all in
. By Corollary 6.1.4 again,
divides , and both are monic
of degree , so
. In particular each root is simple and
has no roots
outside ,
real or complex. This proves the theorem.
So the weight hands us points inside its
interval, chosen by the inner product rather than by us.
The next subsection says what they are good for.
Quick check. The monic Laguerre
polynomial of degree for the weight on is . Check
that Theorem 11.10.5 is
not lying.
Solution. The roots are , that is
and
:
real, distinct, and both in . An unbounded
interval is no obstacle, since the theorem asks only
that the weight be positive with finite moments, and
.
Gaussian
quadrature
Chapter 6 wrote as a fixed combination of , and , valid for every
(Example (Integrals and Slopes from Values)).
The construction works for any prescribed nodes:
integrate the Lagrange interpolation formula (Theorem 6.6.1) and
distinct nodes
give a rule exact on .
But a -point
rule has numbers
at its disposal,
nodes and
coefficients, and only of them were spent.
Spending the nodes too should buy degree . Where to put them
is the question, and the answer is the one place we have
not chosen: the roots of .
Theorem
11.10.6 (Gaussian Quadrature)
Let be a
weight on
with , let , and let be the roots of the monic
orthogonal polynomial , which lie in by Theorem 11.10.5.
Let be the Lagrange
polynomials for these nodes (Definition 6.6.1) and
set Then every is
strictly positive, and for every .
Idea. Two moves, one for each side
of the identity. On the left, divide by : . The degree
bookkeeping — against — forces , which
is exactly the condition under which vanishes. So the integral of
is the integral
of its remainder. On the right, vanishes at every
node, so the sum also only sees the remainder. And the
remainder has degree at most , which is precisely
the range on which nodes reproduce a
polynomial exactly. The two sides meet there.
Proof. Let .
Since , Theorem 6.1.1 gives
with If
then trivially. Otherwise has degree
at most ,
while by Theorem 0.8.3; hence
.
Either way , so Proposition 11.10.2
gives , that is, .
Therefore At each node, , so Since and are
distinct points, Theorem 6.6.1 (b)
gives . Integrating against and using the
definition of , the last step by . Combining
with proves
the quadrature identity.
For positivity, fix and apply the identity
to ,
which is legitimate because . By Theorem 6.6.1 (a),
, so the right-hand side collapses
to , while
the left-hand side is . Since , positive
definiteness gives .
Each hypothesis did one job. Orthogonality to lower
degrees killed the quotient on the left; the nodes being
roots of killed
it again on the right. That is also why the nodes cannot
be moved: shift one and the second cancellation fails.
Exercise (C2: The ceiling is 2k − 1)
shows is a
ceiling, not just what this proof happens to reach.
Example
(Two-point Gauss–Legendre)
Take on
and . Find the nodes and
weights, and test the resulting rule on
and on .
Solution. The monic is (Example (Gram–Schmidt on )),
so the nodes are and . The Lagrange polynomials are and , with .
Since and , and by the same computation with the roles
swapped. The rule is Two evaluations, no coefficients to remember,
every cubic exactly.
The cubic test. Exactly, . By
the rule, and , whose sum is . The odd parts cancel
and the constant and quadratic parts contribute and .
The quartic test. Exactly, . By the rule, . These differ, so degree is already out of
reach. Compare Example (Integrals and Slopes from Values),
where the nodes were fixed in advance and
three of them guaranteed exactness only
up to degree —
guaranteed by the construction, that is; that particular
symmetric rule happens to integrate cubics exactly as
well, which is luck of the symmetry and not something
the construction promises.
Warning. “Exact” means
exact on polynomials. Theorem 11.10.6
says nothing about for a general , and nothing about what
happens as grows;
on a non-polynomial the rule returns an approximation
whose error is a question for analysis. What linear
algebra delivers is exactness on a subspace, and the
reason that subspace is the largest available.
The
classical families
Four weights account for most of the orthogonal
polynomials that occur in practice. The table records
them; the entries are statements, and Exercise (B2: Two rows of the table)
asks you to verify two of the rows by integration.
| Legendre |
|
|
|
|
| Chebyshev |
|
|
|
|
| Hermite |
|
|
|
|
| Laguerre |
|
|
|
|
The last two columns give the monic members of
degrees and ; in every row. The
Chebyshev weight is unbounded at and the Hermite and
Laguerre intervals are unbounded, so in those three rows
the integrals defining the inner product are improper;
they converge, which is the finiteness condition our
definition of a weight demanded. The first three weights
are even on symmetric intervals, so their all vanish and their
members alternate in parity; the Laguerre weight is not,
and there.
Section 9 of this chapter meets the Chebyshev family
again from the Fourier side, where the substitution
turns its awkward weight into .
Exercises
A. Check your
understanding
Exercise
(A1)
- (a) Define what it
means for to be a monic orthogonal
polynomial sequence for an inner product on
.
- (b) State the
three-term recurrence, including the formulas for and and the starting
values.
- (c) True or false:
for
every .
Justify your answer.
- (d) For , how many roots does have, of what kind,
and where?
- (e) True or false:
for every inner product on one has . Justify your answer.
- (f) How many nodes
does a Gaussian rule need in order to integrate every
polynomial of degree at most exactly?
Solution.
(a) A sequence
with
and leading coefficient for every , and
whenever
(Definition 11.10.1).
(b) With , and , one has
for , where and (Theorem 11.10.3).
(c) True. Both
and are strictly positive, since
and are non-zero and
the inner product is positive definite.
(d) Seven roots,
all real, all distinct, all in the open interval (Theorem 11.10.5).
In particular
has no complex roots and no repeated ones.
(e) False. Take
the coefficient inner product on , where and
(Example (A functional with no Riesz vector)).
Then , while . The identity is a property
of the weight and node constructions (Proposition 11.10.1),
not of inner products in general.
(f) Six. A -point rule is exact up
to degree ,
and .
B. Practice
Exercise
(B1: A weighted interval)
On put
. Compute the monic , and , and verify directly
that .
Solution. The moments are , so for .
. Then
, so , and Next , so and Theorem 11.10.3
gives Verification. , and . Since spans , this is
the whole check.
Exercise
(B2: Two rows of the table)
Verify the degree- and degree- entries of the table
for two of the classical families.
- (a) Chebyshev:
on . You may
use, without proof, that and .
- (b) Laguerre: on . You may use,
without proof, that .
Solution.
(a) The weight is
even, so whenever is odd. Hence , and
is monic of
degree and
orthogonal to , so by Proposition 11.10.2.
Write . Then , forcing ; and , forcing . So
.
(b) The moments
are , so for . For : , so . For , the two conditions and
read Subtracting gives , then . So , whose
roots
we already located in the Quick check above.
Solution. Since the weight is even, , and . Hence Verification. , and Orthogonality to and is automatic by
parity.
Exercise
(B4: A two-point rule on the unit interval)
Work on
with .
Exercise B2 of Section 2 (Exercise (B2: Gram–Schmidt on polynomials))
found the monic orthogonal polynomials , , .
- (a) Write down the
two-point Gaussian rule for .
- (b) Check it on
, and show
that it fails on .
Solution.
(a) The nodes are
the roots of , namely both in , as Theorem 11.10.5
promises. With and , and likewise. So for .
(b) Write
with ,
so . The odd powers of cancel in a sum over
and , so and half of that is . For the quartic, so the rule returns , whereas
. The rule is exact to degree and no
further.
C. Going deeper
Exercise
(C1: Interlacing roots)
Let be a
weight on
and let be
its monic orthogonal polynomial sequence. The roots of
are said to
interlace those of when exactly one
root of lies
strictly between each pair of consecutive roots of . This is true for
every ; prove the
first two cases by hand.
- (a) Let be the root of and the roots of
. Prove that
.
- (b) Let be
the roots of .
Prove that .
Hint: evaluate the three-term recurrence at a
root.
Solution.
(a) By Theorem 11.10.3
with , . Evaluating at gives ,
since .
By Theorem 11.10.5,
with real, and this product is negative exactly
on .
Hence .
(b) By Theorem 11.10.3
with , . At a root of the first term dies,
so By (a), and , and , so and .
By Theorem 11.10.5,
with . Such a product is negative on
,
positive on , negative on , and positive
on .
So
puts in or in , and puts
in or in
. The
option
is impossible: it would give ,
where is
positive, contradicting . The
option
is impossible too: it would give ,
where is
negative, contradicting . Hence
and , which is the assertion.
The general case is not proved this way. The
coefficients
and assemble
into a symmetric tridiagonal matrix whose characteristic
polynomial is ,
and the interlacing then follows from the interlacing
theorem for the eigenvalues of a symmetric matrix and
its leading submatrices, which is Chapter 17’s
business.
Exercise
(C2: The ceiling is 2k − 1)
Let be a
weight on
and let .
- (a) Let be
any distinct real numbers
and any real numbers.
Prove that there is an with .
- (b) Hence deduce
that the Gaussian rule of Theorem 11.10.6 is
exact on and on no larger space .
Solution.
(a) Put and , a polynomial of degree . Every node is a root
of , so for every
and the
right-hand side is , whatever the are. The
left-hand side is , and , so by
positive definiteness. The two sides differ.
(b) Exactness on
is Theorem 11.10.6.
Suppose the rule were exact on for some
. Since
, it would then be exact on every
polynomial of degree at most . But its nodes are
distinct real
numbers (Theorem 11.10.5),
so (a) supplies a polynomial of degree on which it is not.
Hence no such
exists, and
is the largest degree for which the rule is exact. The
concrete instance ,
is the failure computed in Example (Two-point Gauss–Legendre).