FS1 7.2 - Mean and variance from generating functions

FS1 7.2 - Mean and variance from generating functions

Differentiating a PGF turns its probability coefficients into moments. You will justify the mean and variance formulae, prove the four standard distributions' moment results and use calculus with less familiar PGFs (FS1 7.2).

Why moments appear at one

Two PGF curves pass through (1,1)(1,1) with the same slope. Must their distributions have the same variance?

Consider GX(t)=tG_X(t)=t, which describes X=1X=1 with certainty, and GW(t)=(1+t2)/2G_W(t)=(1+t^2)/2, which describes equally likely values 00 and 22. Both have value 11 and derivative 11 at t=1t=1. Yet one distribution never varies and the other does. The second derivative will reveal what the shared tangent cannot.

You already know

A PGF is GX(t)=∑xP(X=x)txG_X(t)=\sum_x \mathrm P(X=x)t^x. Coefficients and derivative values at zero recover probabilities; moments use a different input. Recall Var⁡(X)=E(X2)−[E(X)]2\operatorname{Var}(X)=\mathrm E(X^2)-[\mathrm E(X)]^2, and use product and chain rules when differentiating.

For the distributions here, the necessary moments are finite and the PGF can be differentiated near t=1t=1. Write px=P(X=x)p_x=\mathrm P(X=x). Differentiating the series gives

GX′(t)=∑xxpxtx−1differentiate each powerGX′(1)=∑xxpx=E(X)all remaining powers become oneGX′′(1)=∑xx(x−1)pxdifferentiate twice=E(X2)−E(X)expand x(x−1)\begin{aligned} G_X'(t)&=\sum_x xp_xt^{x-1} &&\text{differentiate each power}\\ G_X'(1)&=\sum_x xp_x=\mathrm E(X) &&\text{all remaining powers become one}\\ G_X''(1)&=\sum_x x(x-1)p_x &&\text{differentiate twice}\\ &=\mathrm E(X^2)-\mathrm E(X) &&\text{expand }x(x-1) \end{aligned}

Thus E(X2)=GX′′(1)+GX′(1)\mathrm E(X^2)=G_X''(1)+G_X'(1). Subtract the square of the mean to obtain the variance. This also explains why the second derivative alone is not the variance.

Moments from a PGF

Where the relevant moments exist,

E(X)=GX′(1),Var⁡(X)=GX′′(1)+GX′(1)−[GX′(1)]2\mathrm E(X)=G_X'(1),\qquad \operatorname{Var}(X)=G_X''(1)+G_X'(1)-[G_X'(1)]^2

The intermediate identity is GX′′(1)=E[X(X−1)]G_X''(1)=\mathrm E[X(X-1)]. The mean and variance formulae are supplied; the displayed series argument justifies them.

In the formula booklet.

The two opening curves have the same height and slope at 11, but different bending. Their shared mean therefore leaves room for different variances.

[DIAGRAM: asset_name: Same PGF value and slope with different variance; asset_slug: 9fm0-fs114-shared-tangent; caption: PGFs can share their mean while their second derivatives reveal different spreads; description: A graph shows G X of t equals t in grey and G W of t equals one plus t squared over two in sage, for t from zero to one point five. Both curves meet at the marked point one comma one with the same tangent. The legend states that X is always one with variance zero, whereas W is zero or two equally likely with variance one. A note says both derivatives at one equal one, while the second derivatives at one are zero and one.]
PGFs can share their mean while their second derivatives reveal different spreads

Key idea

Differentiate first, then substitute t=1t=1. The second derivative supplies E[X(X−1)]\mathrm E[X(X-1)]; add the mean before subtracting its square.

Distinguish the three quantities in the following check.

Proving the binomial and Poisson moments

To prove a standard moment result using its PGF, show the derivatives, their values at one and the final variance simplification.

The booklet supplies both PGFs and their moments. A proof must connect them by calculus; quoting the target mean and variance is not a derivation.

Worked example 1: binomial moments from a polynomial power

Question. Use the PGF of X∼B(n,p)X\sim\mathrm B(n,p) to prove that its mean is npnp and its variance is np(1−p)np(1-p).

Plan. Write q=1−pq=1-p, differentiate (q+pt)n(q+pt)^n twice and use q+p=1q+p=1.

GX(t)=(q+pt)nsupplied PGFGX′(t)=np(q+pt)n−1chain ruleGX′′(t)=n(n−1)p2(q+pt)n−2chain rule againGX′(1)=np,GX′′(1)=n(n−1)p2q+p=1\begin{aligned} G_X(t)&=(q+pt)^n &&\text{supplied PGF}\\ G_X'(t)&=np(q+pt)^{n-1} &&\text{chain rule}\\ G_X''(t)&=n(n-1)p^2(q+pt)^{n-2} &&\text{chain rule again}\\ G_X'(1)&=np,\qquad G_X''(1)=n(n-1)p^2 &&q+p=1 \end{aligned}

Hence

E(X)=npfirst derivative at oneVar⁡(X)=n(n−1)p2+np−n2p2supplied variance identity=np−np2=np(1−p)collect terms\begin{aligned} \mathrm E(X)&=np &&\text{first derivative at one}\\ \operatorname{Var}(X)&=n(n-1)p^2+np-n^2p^2 &&\text{supplied variance identity}\\ &=np-np^2=np(1-p) &&\text{collect terms} \end{aligned}

Why this step? Substituting 11 after differentiation makes the bracket equal 11 without losing the powers of pp produced by the chain rule.

Answer. E(X)=np\mathrm E(X)=np, Var⁡(X)=np(1−p)\operatorname{Var}(X)=np(1-p), as required.

Check. At n=1n=1, the second derivative is zero and the variance reduces to the Bernoulli value p(1−p)p(1-p). The degenerate case n=0n=0 has both moments zero directly.

Worked example 2 changes the count model from binomial to Poisson. The derivative-at-one procedure stays the same; the exponential replaces the polynomial power.

Worked example 2: Poisson moments from an exponential

Question. Use the PGF of Y∼Po⁡(λ)Y\sim\operatorname{Po}(\lambda) to prove that its mean and variance both equal λ\lambda.

Plan. Differentiate the exponential twice, then evaluate at one before simplifying.

GY(t)=eλ(t−1)supplied PGFGY′(t)=λeλ(t−1)chain ruleGY′′(t)=λ2eλ(t−1)differentiate againGY′(1)=λ,GY′′(1)=λ2e0=1Var⁡(Y)=λ2+λ−λ2=λvariance identity\begin{aligned} G_Y(t)&=\mathrm e^{\lambda(t-1)} &&\text{supplied PGF}\\ G_Y'(t)&=\lambda\mathrm e^{\lambda(t-1)} &&\text{chain rule}\\ G_Y''(t)&=\lambda^2\mathrm e^{\lambda(t-1)} &&\text{differentiate again}\\ G_Y'(1)&=\lambda,\qquad G_Y''(1)=\lambda^2 &&\mathrm e^0=1\\ \operatorname{Var}(Y)&=\lambda^2+\lambda-\lambda^2=\lambda &&\text{variance identity} \end{aligned}

Why this step? Each differentiation contributes another factor λ\lambda. The cancellation in the variance occurs only after adding the mean and subtracting its square.

Answer. E(Y)=λ\mathrm E(Y)=\lambda and Var⁡(Y)=λ\operatorname{Var}(Y)=\lambda, as required.

Check. For λ=3\lambda=3, the derivative values are 33 and 99, giving variance 9+3−9=39+3-9=3. These agree with the supplied Poisson moments.

Both pairs of moment results are in the formula booklet. The proofs explain their structure: the binomial variance retains a factor 1−p1-p, while the Poisson variance equals its mean.

Watch out

Wrong: differentiate (q+pt)n(q+pt)^n as n(q+pt)n−1n(q+pt)^{n-1}. Right: include the inner derivative pp, then include another factor pp on the second differentiation. In a proof, unsupported target formulae do not replace these steps.

Use the exponential method with a specified parameter in this check.

Proving the waiting-time moments

Keep the successful-trial factor in a waiting-time PGF when differentiating: removing it changes the variable and its mean.

Write q=1−pq=1-p, with 0<p≤10<p\le1. The geometric trial-count PGF is pt/(1−qt)pt/(1-qt). Its first derivative simplifies neatly if you combine the numerator terms before differentiating again.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account