Download Summation and Series: Definitions, Computing, & Strategies for Asymptotic Estimations and more Lecture notes Calculus in PDF only on Docsity!
Notes on summations and related topics
James Aspnes
December 13, 2010
1 Summations
Summations are the discrete versions of integrals; given a sequence xa, xa+1,... , xb, its sum xa + xa+1 + · · · + xb is written as
∑b i=a xi. The large jagged symbol is a stretched-out version of a capital Greek letter sigma. The variable i is called the index of summation, a is the lower bound or lower limit, and b is the upper bound or upper limit. Mathematicians invented this notation centuries ago because they didn’t have for loops; the intent is that you loop through all values of i from a to b (including both endpoints), summing up the body of the summation for each i. If b < a, then the sum is zero. For example,
∑^ −^5
i=
2 i^ sin i i^3
This rule mostly shows up as an extreme case of a more general formula, e.g. ∑^ n
i=
i =
n(n + 1) 2
which still works even when n = 0 or n = −1 (but not for n = −2).
∑Summation notation is used both for laziness (it’s more compact to write n i=0(2i^ + 1) than 1 + 3 + 5 + 7 +^ · · ·^ + (2n^ + 1)) and precision (it’s also more clear exactly what you mean).
1.1 Formal definition
For finite sums, we can formally define the value by either of two recurrences:
∑^ b
i=a
f (i) =
0 if b < a f (a) +
∑b i=a+1 f^ (i)^ otherwise.^
∑^ b
i=a
f (i) =
0 if b < a f (b) +
∑b− 1 i=a f^ (i)^ otherwise.^
In English, we can compute a sum recursively by computing either the sum of the last n − 1 values or the first n − 1 values, and then adding in the value we left out. (For infinite sums we need a different definition; see below.)
1.2 Choosing and replacing index variables
When writing a summation, you can generally pick any index variable you like, although i, j, k, etc., are popular choices. Usually it’s a good idea to pick an index that isn’t used outside the sum. Though
∑^ n
n=
n =
∑^ n
i=
i
has a well-defined meaning, the version on the right-hand side is a lot less confusing. In addition to renaming indices, you can also shift them, provided you shift the bounds to match. For example, rewriting
∑^ n
i=
(i − 1)
as n∑− 1
j=
j
(by substituting j for i − 1) makes the sum more convenient to work with.
1.3 Scope
The scope of a summation extends to the first addition or subtraction symbol that is not enclosed in parentheses or part of some larger term (e.g., in the numerator of a fraction). So
∑^ n
i=
i^2 + 1 =
( (^) n ∑
i=
i^2
∑^ n
i=
i^2 6 =
∑^ n
i=
(i^2 + 1).
Since this can be confusing, it is generally safest to wrap the sum in paren- theses (as in the second form) or move any trailing terms to the beginning. An exception is when adding together two sums, as in
∑^ n
i=
i^2 +
∑^ n^2
i=
i =
( (^) n ∑
i=
i^2
∑^ n^2
i=
i
Here the looming bulk of the second sigma warns the reader that the first sum is ending; it is much harder to miss than the relatively tiny plus symbol in the first example.
If S is infinite, this is trickier. For countable S, where there is a bijection f : N ↔ S, we can sometimes rewrite
∑
i∈S
xi =
∑^ ∞
i=
xf (i).
and use the definition of an infinite sum (given below). Note that if the xi have different signs the result we get may depend on which bijection we choose. For this reason such infinite sums are probably best avoided unless you can explicitly use N as the index set.
1.5 Sums without explicit bounds
When the index set is understood from context, it is often dropped, leaving only the index, as in
i i
(^2). This will generally happen only if the index spans all
possible values in some obvious range, and can be a mark of sloppiness in formal mathematical writing. Theoretical physicists adopt a still more lazy approach, and leave out the
i part entirely in certain special types of sums: this is known as the Einstein summation convention after the notoriously lazy physicist who proposed it.
1.6 Infinite sums
Sometimes you may see an expression where the upper limit is infinite, as in
∑^ ∞
i=
i^2
The meaning of this expression is the limit of the series s obtained by taking the sum of the first term, the sum of the first two terms, the sum of the first three terms, etc. The limit converges to a particular value x if for any > 0, there exists an N such that for all n > N , the value of sn is within of x (formally, |sn − x| < ). We will see some examples of infinite sums when we look at generating functions.
1.7 Double sums
Nothing says that the expression inside a summation can’t be another sum- mation. This gives double sums, such as in this rather painful definition of multiplication for non-negative integers:
a × b def =
∑^ a
i=
∑^ b
j=
If you think of a sum as a for loop, a double sum is two nested for loops. The effect is to sum the innermost expression over all pairs of values of the two indices.
Here’s a more complicated double sum where the limits on the inner sum depend on the index of the outer sum:
∑^ n
i=
∑^ i
j=
(i + 1)(j + 1).
When n = 1, this will compute (0+1)(0+1)+(1+1)(0+1)+(1+1)(1+1) = 7. For larger n the number of terms grows quickly. There are also triple sums, quadruple sums, etc.
2 Computing sums
When confronted with some nasty sum, it is nice to be able to convert into a simpler expression that doesn’t contain any summation signs. It is not always possible to do this, and the problem of finding a simpler expression for a sum is very similar to the problem of computing an integral (see HowToIntegrate): in both cases the techniques available are mostly limited to massaging the sum- mation until it turns into something whose simpler expression you remember. To do this, it helps to both (a) have a big toolbox of sums with known values, and (b) have some rules for manipulating summations to get them into a more convenient form. We’ll start with the toolbox.
2.1 Some standard sums
Here are the three formulas you should either memorize or remember how to derive:
∑^ n
i=
1 = n
∑^ n
i=
i = n(n + 1) 2 ∑^ n
i=
ri^ =
1 − rn+ 1 − r
Rigorous proofs of these can be obtained by induction on n. For not so rigorous proofs, the second identity can be shown (using a trick alleged to have been invented by the legendary 18th-century mathematician Carl Friedrich Gauss at a frighteningly early age by adding up two copies of the sequence running in opposite directions, one term at a time:
S = 1 + 2 + 3 + .... + n S = n + n-1 + n-2 + .... + 1
2S = (n+1) + (n+1) + (n+1) + .... + (n+1) = n(n+1),
Products of sums can be turned into double sums of products and vice versa: ( ∑
i∈S
xi
j∈T
yj
i∈S
j∈T
xiyj.
These identities can often be used to transform a sum you can’t solve into something simpler.
2.3 What to do if nothing else works
If nothing else works, you can try using the ”guess but verify” method, which is a variant on the same method for identifying sequences. Here we write out the values of the sum for the first few values of the upper limit (for example), and hope that we recognize the sequence. If we do, we can then try to prove that a formula for the sequence of sums is correct by induction. Example: Suppose we want to compute
S(n) =
∑^ n
k=
(2k − 1)
but that it doesn’t occur to us to split it up and use the
∑n k=1 k^ and^
∑n k=1 1 formulas. Instead, we can write down a table of values: n S(n) 0 0 1 1 2 1 + 3 = 4 3 1 + 3 + 5 = 9 4 1 + 3 + 5 + 7 = 16 5 1 + 3 + 5 + 7 + 9 = 25 At this point we might guess that S(n) = n^2. To verify this, observe that it holds for n = 0, and for larger n we have S(n) = S(n − 1) + (2n − 1) = (n − 1)^2 + 2n − 1 = n^2 − 2 n + 1 − 2 n − 1 = n^2. So we can conclude that our guess was correct. If this doesn’t work, you could always try using generating functions.
2.4 Strategies for asymptotic estimates
Mostly in algorithm analysis we do not need to compute sums exactly, because we are just going to wrap the result up in some asymptotic expression anyway (see asymptotic notation). This makes our life much easier, because we only need an approximate solution. Here’s my general strategy for computing sums:
2.4.1 Pull out constant factors
Pull as many constant factors out as you can (where constant in this case means anything that does not involve the summation index). Example:
∑n i=
n i = n
∑n i=
1 i =^ nHn^ = Θ(n^ log^ n).^ (See harmonic series below.)
2.4.2 Bound using a known sum
See if it’s bounded above or below by some other sum whose solution you already know. Good sums to try (you should memorize all of these):
Geometric series
∑n i=0 x
i (^) = 1 −xn+ 1 −x and^
i=0 x
i (^) = 1 1 −x. The way to recognize a geometric series is that the ratio between adjacent terms is constant. If you memorize the second formula, you can rederive the first one. If you’re Gauss, you can skip memorizing the second formula. A useful trick to remember for geometric series is that if x is a constant that is not exactly 1∑ , the sum is always big-Theta of its largest term. So for example n i=1 2
i (^) = Θ(2n) (the exact value is 2n+1 (^) − 2), and ∑n i=1 2
−i (^) = Θ(1) (the exact
value is 1 − 2 −n). This fact is the basis of the Master Theorem, described in SolvingRecurrences. If the ratio between terms equals 1, the formula doesn’t work; instead, we have a constant series (see below).
Constant series
∑n i=1 1 =^ n.
Arithmetic series The simplest arithmetic series is
∑n i=1 i^ =^
n(n+1) 2.^ The way to remember this formula is that it’s just n times the average value (n+1)/2. The way to recognize an arithmetic series is that the difference between adjacent terms is constant. The general arithmetic series is of the form
∑n ∑n i=1(ai^ +^ b) = i=1 ai+
∑n i=1 b^ =^ an(n+1)/2+bn.^ Because the general series expands so easily to the simplest series, it’s usually not worth memorizing the general formula.
Harmonic series
∑n i=1 1 /i^ =^ Hn^ = Θ(n^ log^ n). Can be rederived using the integral technique given below or by summing the last half of the series, so this is mostly useful to remember in case you run across H,,n,, (the ”n-th harmonic number”).
2.4.3 Bound part of the sum
See if there’s some part of the sum that you can bound. For example,
∑n i=1 i 3
has a (painful) exact solution, or can be approximated by the integral trick described below, but it can very quickly be solved to within a constant factor by observing that
∑n i=1 i (^3) ≤ ∑n i=1 n (^3) = O(n (^4) ) and ∑n i=1 i (^3) ≥ ∑n i=n/ 2 i
∑n i=n/ 2 (n/2)
(^3) = Ω(n (^4) ).
The other difference is that while an empty sum is defined to have the value 0, an empty product is defined to have the value 1. The reason for this rule (in both cases) is that an empty sum or product should return the identity element for the corresponding operation—the value that when added to or multiplied by some other value x doesn’t change x. This allows writing general rules like: ∑
i∈A
f (i) +
i∈B
f (i) =
i∈A∪B
f (i) ( ∏
i∈A
f (i)
i∈B
f (i)
i∈A∪B
f (i)
which holds as long as A ∩ B = ∅. Without the rule that the sum of an empty set was 0 and the product 1, we’d have to put in a special case for when one or both of A and B were empty. Note that a consequence of this definition is that 0! = 1.
4 Other big operators
Some more obscure operators also allow you to compute some aggregate over a series, with the same rules for indices, lower and upper limits, etc., as
∏ and
. These include: - Big AND: ∧
x∈S
P (x) ≡ P (x 1 ) ∧ P (x 2 ) ∧... ≡ ∀x ∈ S : P (x).
x∈S
P (x) ≡ P (x 1 ) ∨ P (x 2 ) ∨... ≡ ∃x ∈ S : P (x).
i=
Ai = A 1 ∩ A 2 ∩... ∩ An.
i=
Ai = A 1 ∪ A 2 ∪... ∪ An.
These all behave pretty much the way one would expect. One issue that is not obvious from the definition is what happens with an empty index set. Here the rule as with sums and products is to return the identity element for the operation. This will be True for AND, False for OR, and the empty set for union; for intersection, there is no identity element in general, so the intersection over an empty collection of sets is undefined.
