4.5 - Probability generating functions | Cambridge A-Level Further Mathematics

A probability distribution can be stored in a single algebraic expression: the powers record the possible values, and the coefficients record their probabilities. You will construct these probability generating functions, derive the standard forms and recover probabilities without confusing a coefficient with a function value.

Encoding probabilities

How can one polynomial store both the possible values of a random variable and their probabilities? Suppose a count $X$ takes the values $0,1,3$ with probabilities $0.2,0.5,0.3$ . Attach each probability to a power whose exponent is the corresponding value:

0.2t^0+0.5t^1+0.3t^3=0.2+0.5t+0.3t^3.

The missing term in $t^2$ records $\mathrm P(X=2)=0$ . The symbol $t$ is a variable used to organise the information; it is not an observation of $X$ .

Probability generating function

For a nonnegative integer-valued random variable $X$ , with $p_r=\mathrm P(X=r)$ , its probability generating function, or PGF, is

G_X(t)=\mathrm E(t^X)=\sum_{r=0}^{\infty}p_rt^r.

For finite support, the sum is a polynomial. A distribution on $1,2,3,\ldots$ is included: its coefficient $p_0$ is zero.

The definition $G_X(t)=\mathrm E(t^X)$ is in the formula booklet.

A PGF uses integer powers starting at zero. A variable that can take negative values or arbitrary real values does not have a PGF in this sense. At $t=0$ , the constant term gives $G_X(0)=p_0$ ; here the constant monomial $t^0$ is interpreted as $1$ .

Worked example 1: a gap in the possible values

Question. The distribution of $X$ is given by

$r$	$0$	$1$	$3$
$\mathrm P(X=r)$	$0.2$	$0.5$	$0.3$

Find $G_X(t)$ .

Plan. Make each value of $X$ an exponent, and use its probability as the coefficient.

\begin{aligned} G_X(t)&=\sum_r \mathrm P(X=r)t^r \\ &=0.2t^0+0.5t+0.3t^3 \\ &=0.2+0.5t+0.3t^3 \end{aligned}

Answer. $G_X(t)=0.2+0.5t+0.3t^3$ .

Check. The coefficients are nonnegative and add to $1$ . Also, $G_X(-1)=0.2-0.5-0.3=-0.6$ : this agrees with the even-value probability minus the odd-value probability, $0.2-0.8$ .

Now add one to every possible value. The probabilities stay fixed, so only the exponents change.

Worked example 2: shifting the count

Question. A random variable $Y$ takes values $1,2,4$ with probabilities $0.2,0.5,0.3$ , respectively. Find $G_Y(t)$ , and relate it to the PGF in Worked example 1.

Plan. Encode the new values, then factor out the common power of $t$ .

\begin{aligned} G_Y(t)&=0.2t+0.5t^2+0.3t^4 \\ &=t(0.2+0.5t+0.3t^3) \\ &=tG_X(t) \end{aligned}

Multiplying by $t$ shifts every exponent up by one, just as adding one shifts every outcome up by one.

Answer. $G_Y(t)=0.2t+0.5t^2+0.3t^4=tG_X(t)$ .

Check. $G_Y(0)=0$ , matching the absence of a zero outcome. Its odd-value probability is $0.2$ and its even-value probability is $0.8$ , so $G_Y(-1)=0.8-0.2=0.6$ , as the polynomial gives.

The location of a coefficient matters as much as its value. Write zero coefficients mentally when there are gaps.

Watch out

The power is the value of the random variable, not the position of that value in a table. Values $0,2,5$ require powers $t^0,t^2,t^5$ , not $t^0,t^1,t^2$ .

Construct a new PGF with two gaps in its support.

Standard distribution PGFs

Standard distributions lead to standard algebraic forms, because their probability formulae are already the coefficients of familiar series. Recall the probability mass functions for the binomial, geometric and Poisson distributions; these are supplied in the formula booklet.

Use a positive integer $m$ for a uniform distribution, a nonnegative integer $n$ and $0\le p\le1$ for a binomial distribution, $0<p\le1$ for a geometric distribution, and $\lambda>0$ for a Poisson distribution.

For a discrete uniform variable on $1,2,\ldots,m$ , each probability is $1/m$ , so

\begin{aligned} G_X(t)&=\frac{t+t^2+\cdots+t^m}{m} \\ &=\frac{t(1-t^m)}{m(1-t)}\qquad (t\ne1) \end{aligned}

This is a finite geometric sum. At $t=1$ , use the polynomial to obtain $G_X(1)=1$ ; the fraction has a removable gap there. If the uniform values are instead $0,1,\ldots,m-1$ , omit the leading factor $t$ .

For $X\sim\mathrm B(n,p)$ , put $q=1-p$ . The binomial theorem gives

\begin{aligned} G_X(t)&=\sum_{r=0}^{n}\binom nr p^rq^{n-r}t^r \\ &=\sum_{r=0}^{n}\binom nr (pt)^rq^{n-r} \\ &=(q+pt)^n \end{aligned}

For $X\sim\operatorname{Geo}(p)$ , $X$ counts the trial of the first success: its possible values are $1,2,\ldots$ , and $\mathrm P(X=r)=pq^{r-1}$ . Therefore

\begin{aligned} G_X(t)&=pt+pq t^2+pq^2t^3+\cdots \\ &=pt\sum_{j=0}^{\infty}(qt)^j \\ &=\frac{pt}{1-qt}\qquad (|qt|<1) \end{aligned}

For $0<p<1$ , this convergence interval contains $t=1$ . If $p=1$ , $X=1$ with certainty and $G_X(t)=t$ .

Finally, for $X\sim\operatorname{Po}(\lambda)$ , the exponential series gives

\begin{aligned} G_X(t)&=\sum_{r=0}^{\infty}\mathrm e^{-\lambda}\frac{\lambda^r}{r!}t^r \\ &=\mathrm e^{-\lambda}\sum_{r=0}^{\infty}\frac{(\lambda t)^r}{r!} \\ &=\mathrm e^{\lambda(t-1)} \end{aligned}

The exponential series converges for every real $t$ . For an infinite PGF in general, $0\le t\le1$ is safe because the probability series converges there; a stated closed form may have a wider domain.

Standard PGFs

\begin{array}{c|c} \text{Distribution}&G_X(t)\\ \hline \text{Uniform on }1,\ldots,m&\displaystyle\frac1m\sum_{r=1}^{m}t^r\\[3pt] \mathrm B(n,p)&(1-p+pt)^n\\[3pt] \operatorname{Geo}(p)\text{ on }1,2,\ldots&\displaystyle\frac{pt}{1-(1-p)t}\\[6pt] \operatorname{Po}(\lambda)&\mathrm e^{\lambda(t-1)} \end{array}

The PGF forms are not listed in the formula booklet: derive them from the definition and the supplied probability formulae and series.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account

Last Accessed

Further Pure Mathematics 1

Further Pure Mathematics 2

Further Mechanics

Further Probability and Statistics

4.5 - Probability generating functions

Encoding probabilities

Standard distribution PGFs

Continue this lesson.

Learn faster with Nova