5.02a–5.02c - Discrete distributions, expectation and variance

5.02a–5.02c - Discrete distributions, expectation and variance

Build a finite discrete probability distribution, calculate its expectation and variance, and track what happens when its values are shifted or scaled. These tools turn a list of possible outcomes into a model you can calculate with and criticise.

Constructing a distribution

How can a game paying only 00, 11 or 22 units average 1.11.1 units per play? An average need not be a possible outcome. To understand how that happens, first keep a complete account of the possible values and their probabilities.

You already know

For independent events, multiply probabilities along a sequence. Add probabilities of mutually exclusive ways to obtain the same outcome. A probability lies between 00 and 11.

A random variable attaches a numerical value to each outcome of a random experiment. A discrete random variable has a finite or countably infinite set of possible values; here, non-standard distributions will be finite. The support is the set of values with positive probability. The letters distinguish the variable XX from one possible value xx.

Discrete probability distribution

A discrete probability distribution specifies every possible value xx of a random variable XX and its probability p(x)=P(X=x)p(x)=\mathrm{P}(X=x). Each probability is non-negative and the probabilities sum to 11. For example, probabilities 14,12,14\tfrac14,\tfrac12,\tfrac14 on 0,1,20,1,2 give a distribution; probabilities 14,12,12\tfrac14,\tfrac12,\tfrac12 do not.

A table and a probability function are two ways to give the same information. Values absent from the support have probability zero. A probability function must state its support, otherwise its meaning is incomplete. The non-negativity and total-probability conditions are not listed as a separate result in the formula booklet: learn them.

Worked example 1: collecting equally likely outcomes

Question. Two independent fair coins are tossed. Let XX be the number of heads. Construct the distribution of XX.

Plan. List the equally likely sequences, then collect those giving the same number of heads.

{TT,TH,HT,HH}four sequencesP(X=0)=14TT onlyP(X=1)=24=12TH or HTP(X=2)=14HH only\begin{aligned} \{\mathrm{TT},\mathrm{TH},\mathrm{HT},\mathrm{HH}\}& &&\text{four sequences}\\ \mathrm{P}(X=0)&=\tfrac14 &&\text{TT only}\\ \mathrm{P}(X=1)&=\tfrac24=\tfrac12 &&\text{TH or HT}\\ \mathrm{P}(X=2)&=\tfrac14 &&\text{HH only} \end{aligned}

Why this step? The values 0,1,20,1,2 are not equally likely: there are two ways to obtain one head.

Answer.

xx001122
P(X=x)\mathrm{P}(X=x)14\tfrac1412\tfrac1214\tfrac14

Check. 14+12+14=1\tfrac14+\tfrac12+\tfrac14=1, with no possible value omitted.

Change just the chance of a head to 34\tfrac34, keeping two independent tosses. The possible values stay the same; the sequence probabilities change.

Worked example 2: unequal sequence probabilities

Question. Each of two independent tosses has probability 34\tfrac34 of a head. Construct the distribution of the number XX of heads, giving a probability function.

Plan. Multiply the head and tail probabilities for each sequence and combine the two one-head sequences.

P(X=0)=(14)2=116two tailsP(X=1)=2(34)(14)=616two ordersP(X=2)=(34)2=916two heads\begin{aligned} \mathrm{P}(X=0)&=\left(\tfrac14\right)^2=\tfrac1{16} &&\text{two tails}\\ \mathrm{P}(X=1)&=2\left(\tfrac34\right)\left(\tfrac14\right)=\tfrac6{16} &&\text{two orders}\\ \mathrm{P}(X=2)&=\left(\tfrac34\right)^2=\tfrac9{16} &&\text{two heads} \end{aligned}

Why this step? Independence allows multiplication within a sequence; the two distinct orders require addition.

Answer. p(0)=116p(0)=\tfrac1{16}, p(1)=616p(1)=\tfrac6{16}, p(2)=916p(2)=\tfrac9{16}, and p(x)=0p(x)=0 otherwise.

Check. (1+6+9)/16=1(1+6+9)/16=1; the most likely value is now 22, as expected from the high head probability.

The list of possible values is separate from the mechanism that makes them likely.

Watch out

Wrong: “There are three possible values, so each has probability 13\tfrac13.” Equal likelihood needs justification from the experiment. Count or calculate the ways producing each value first.

Try changing only the probability of a head once more.

Expectation as a weighted average

An expectation weights each possible value by how likely it is. A value occurring three times as often contributes three times as much to the long-run average; it does not simply get one vote in an unweighted mean.

For a game paying 00, 11 or 22 units with probabilities 0.20.2, 0.50.5, 0.30.3, imagine 100100 plays with exactly the expected proportions. The total payment would be 0(20)+1(50)+2(30)=1100(20)+1(50)+2(30)=110 units, giving 1.11.1 units per play. Actual frequencies fluctuate, but repeated independent play makes this weighted average the relevant target.

Expectation

If XX takes values xix_i with probabilities pip_i, then

μ=E(X)=∑ixipi\mu=\mathrm{E}(X)=\sum_i x_i p_i

The sum is over the entire support.
In the formula booklet.

The expectation has the same units as XX. It need not be a possible outcome: 1.11.1 is an average payment, not a newly invented fourth prize.

Worked example 3: a mean outside the support

Question. The distribution of XX is P(X=0)=0.2\mathrm{P}(X=0)=0.2, P(X=1)=0.5\mathrm{P}(X=1)=0.5, P(X=2)=0.3\mathrm{P}(X=2)=0.3. Find E(X)\mathrm{E}(X).

Plan. Multiply each value by its probability and sum.

E(X)=0(0.2)+1(0.5)+2(0.3)weight each value=0+0.5+0.6=1.1add contributions\begin{aligned} \mathrm{E}(X)&=0(0.2)+1(0.5)+2(0.3) &&\text{weight each value}\\ &=0+0.5+0.6=1.1 &&\text{add contributions} \end{aligned}

Why this step? The probabilities supply the weights; dividing by the number of values would count those weights twice.

Answer. E(X)=1.1\mathrm{E}(X)=1.1.

Check. The answer lies between 00 and 22, and slightly above 11, since the upper value is more likely than the lower one.

Change only the largest value from 22 to 44; keep its probability at 0.30.3. The mean increases even though no probability changes.

Worked example 4: changing one possible value

Question. A random variable WW takes values 0,1,40,1,4 with probabilities 0.2,0.5,0.30.2,0.5,0.3, respectively. Find E(W)\mathrm{E}(W).

Plan. Use the new value in the weighted sum, leaving the probabilities unchanged.

E(W)=0(0.2)+1(0.5)+4(0.3)use the actual values=0.5+1.2=1.7sum\begin{aligned} \mathrm{E}(W)&=0(0.2)+1(0.5)+4(0.3) &&\text{use the actual values}\\ &=0.5+1.2=1.7 &&\text{sum} \end{aligned}

Why this step? Moving probability 0.30.3 upwards by 22 adds 2(0.3)=0.62(0.3)=0.6 to the mean.

Answer. E(W)=1.7\mathrm{E}(W)=1.7.

Check. 1.7−1.1=0.61.7-1.1=0.6, agreeing with the change in the last contribution.

A complete expectation calculation includes values that are negative or zero as well as positive ones.

Watch out

Wrong: E(X)=(0+1+2)/3\mathrm{E}(X)=(0+1+2)/3. That assumes equal probabilities. Right: E(X)=0(0.2)+1(0.5)+2(0.3)=1.1\mathrm{E}(X)=0(0.2)+1(0.5)+2(0.3)=1.1.

The next distribution changes the top value once more, so you can see exactly which contribution changes.

Measuring spread with variance

The mean gives a location, while variance measures spread about it. Squaring deviations makes departures on both sides count positively; otherwise positive and negative deviations would cancel.

Variance

For μ=E(X)\mu=\mathrm{E}(X),

Var(X)=∑i(xi−μ)2pi=E(X2)−μ2=∑ixi2pi−μ2\begin{aligned} \mathrm{Var}(X)&=\sum_i (x_i-\mu)^2p_i\\ &=\mathrm{E}(X^2)-\mu^2=\sum_i x_i^2p_i-\mu^2 \end{aligned}

Variance is non-negative and has squared units; the standard deviation is Var(X)\sqrt{\mathrm{Var}(X)}, in the units of XX.
In the formula booklet.

Both variance expressions describe the same quantity; no proof of the identities is needed here. Notice the different operations in E(X2)\mathrm{E}(X^2) and (E(X))2(\mathrm{E}(X))^2: square the individual values for the first, and square their weighted average for the second.

Worked example 5: using the second moment

Question. For XX taking 0,1,20,1,2 with probabilities 0.2,0.5,0.30.2,0.5,0.3, find its variance.

Plan. Calculate E(X2)\mathrm{E}(X^2), then subtract the square of E(X)=1.1\mathrm{E}(X)=1.1.

E(X2)=02(0.2)+12(0.5)+22(0.3)square the values=1.7sumVar(X)=1.7−1.12=0.49subtract mean squared\begin{aligned} \mathrm{E}(X^2)&=0^2(0.2)+1^2(0.5)+2^2(0.3) &&\text{square the values}\\ &=1.7 &&\text{sum}\\ \mathrm{Var}(X)&=1.7-1.1^2=0.49 &&\text{subtract mean squared} \end{aligned}

Why this step? A second moment includes location as well as spread; subtracting μ2\mu^2 leaves the spread about the mean.

Answer. Var(X)=0.49\mathrm{Var}(X)=0.49.

Check. The direct expression gives 1.12(0.2)+0.12(0.5)+0.92(0.3)=0.491.1^2(0.2)+0.1^2(0.5)+0.9^2(0.3)=0.49.

Change only the top value from 22 to 44, as before. Recalculate the mean as well: measuring spread about the old mean would answer a different question.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account