2b, 2c - Exploring the 2027 large data set

2b, 2c - Exploring the 2027 large data set

Use the prescribed LDS9 health data to build an enquiry you can defend: understand the measurements, select relevant complete rows, and use software summaries and graphs. You will compare repeated samples with the full available data and interpret a supplied simulation test without treating a historical sample as today's population.

LDS9 is a historical sample of people

Why does a person with systolic readings 104104, 104104 and 112112 have a supplied average of 108108, rather than 106.7106.7? The answer is in the measurement rule, not a mysterious new way to add. This is the first real person in the workbook, and the rule will matter whenever you analyse its blood-pressure columns.

You already know

A mean adds the included measurements and divides by their number. A sample and its target population are different things; the selection, place and time affect the conclusions you can draw.

For the H640 June 20272027 examination, the prescribed resource is LDS9, with its student notes and source information. It contains a random sample of 200200 people aged 1616 or over from the American National Health and Nutrition Examination Survey, NHANES, in 20032003–20042004. Read the Information sheet. These public-domain survey data were downloaded for the resource in 20172017. LDS10 belongs to later H640 examinations, not June 20272027.

One Data row is one person. Age is in years; Weight is in kg; Height, waist and limb lengths are in cm; Pulse counts beats over 6060 seconds; blood pressures are in mmHg. Sex, Marital, Food30 and Arm are categorical fields. Marital categories have no numerical rank that would justify calculating a correlation from arbitrary category numbers.

Recorded measurement and derived variable

A recorded measurement is an observed value, such as height. A derived variable is calculated from other measurements, such as a mean blood pressure or BMI. Its definition determines which inputs and units to use.

For example, the first person's mass 60.3 kg60.3\,\mathrm{kg} and height 173.4 cm=1.734 m173.4\,\mathrm{cm}=1.734\,\mathrm m give BMI 60.3/1.7342=20.0548…60.3/1.734^2=20.0548\ldots, agreeing with the supplied 20.0520.05. BMI uses mass in kg divided by squared height in metres; the centimetre value must be converted. This definition is supplied by the context, not printed in the formula sheet.

For blood pressure, interviewers sought three measurements at least one minute apart, with at most four attempts. The supplied Systolic and Diastolic averages ignore the first reading unless it is the only reading. Here “first reading” means the first successfully recorded measurement: if attempt one is missing, it is not necessarily the first column. Average the remaining recorded readings, not missing attempts. The Data headers misspell the individual diastolic fields as “Disatolic1” to “Disatolic4”; retain those labels when locating source columns, while writing “diastolic” in your conclusions.

Worked example 1: the first person's average

Question. Data row 22 has systolic attempts 104,104,112104,104,112, with the fourth missing. Determine the average required by the Information sheet.

Plan. Exclude the first reading because later readings exist, then average the two available later values.

included sum=104+112=216use attempts 2 and 3average=2162=108divide by two observations\begin{aligned} \text{included sum}&=104+112=216 && \text{use attempts }2\text{ and }3\\ \text{average}&=\frac{216}{2}=108 && \text{divide by two observations} \end{aligned}

Why this step? The first reading is deliberately excluded by the recording protocol; the missing fourth attempt contributes neither a value nor an extra divisor.

Answer. The prescribed systolic average is 108 mmHg108\,\mathrm{mmHg}.

Check. The result lies halfway between the two included readings and matches the supplied Systolic value.

Change only the person in Worked example 2. The measurement protocol and missing-fourth-attempt pattern stay the same.

Worked example 2: another real row

Question. Data row 44 has systolic attempts 142,138,140142,138,140, with the fourth missing. Determine its prescribed average.

Plan. Apply the same exclusion rule before calculating the mean.

included sum=138+140=278omit the first attemptaverage=2782=139two later observations\begin{aligned} \text{included sum}&=138+140=278 && \text{omit the first attempt}\\ \text{average}&=\frac{278}{2}=139 && \text{two later observations} \end{aligned}

Why this step? A standard spreadsheet average of all three numeric attempts would include a value that this variable's definition excludes.

Answer. The prescribed systolic average is 139 mmHg139\,\mathrm{mmHg}.

Check. It is between 138138 and 140140, and agrees with the supplied average.

If only the first reading exists, retain it instead: discarding it would discard the person's only measurement. No blood-pressure averaging protocol or arithmetic-mean formula is printed in the formula sheet; learn to apply the supplied definition.

Watch out

Wrong: the code #N/A means a blood pressure of zero. Right: it means the measurement is unavailable. The code was included specifically to stop software interpreting a blank as zero.

A third real row gives you the same calculation without copying either example.

Complete observations are selected for named variables

Choose the variables before removing unavailable values; a person with a missing arm length can still contribute to an age–blood-pressure enquiry.

Keep the source unchanged. In a working copy, retain the original Data row number as an identifier, and select Age with the measurement you need. A spreadsheet column holds a field, a row holds a person's observations, and a cell such as U2 identifies a particular entry. Delete or exclude a whole selected analysis row when a required member of its pair is missing; never shift individual cells upward, which would pair different people.

For Age against Systolic, original columns B and U must both be numeric. All 200200 ages are present, but 1414 Systolic values are missing, leaving 186186 complete pairs. A missing fourth attempt does not invalidate an available supplied average. Compact the selected pairs into a separate analysis table: merely hiding rows does not make ordinary spreadsheet summary functions ignore them.

MEI sample standard deviation

For n>1n>1 numerical observations with mean xˉ\bar x,

Sxx=∑xi2−(∑xi)2n,s=Sxxn−1S_{xx}=\sum x_i^2-\frac{(\sum x_i)^2}{n},\qquad s=\sqrt{\frac{S_{xx}}{n-1}}

The sample variance uses divisor n−1n-1, and ss has the original measurement's units. In the formula sheet.

On a compact table with row 11 containing headers and the selected measurement in column C, use COUNT\texttt{COUNT}, AVERAGE\texttt{AVERAGE} and STDEV.S\texttt{STDEV.S} on the explicit data range. The letter S in this function is a useful reminder of the sample divisor; check your calculator's output definition rather than guessing from its label.

Worked example 3: summarising the complete systolic data

Question. The 186186 complete Age–Systolic pairs have ∑x=22779\sum x=22779 and ∑x2=2856003\sum x^2=2856003 for Systolic. Determine the mean and sample SD to 33 decimal places.

Plan. Use the selected count for both the mean and the sample-variance calculation.

xˉ=22779186=122.467741…mean of available valuesSxx=2856003−227792186=66310.30645…centred sum of squaress=66310.30645…185=18.932355…sample divisor n−1\begin{aligned} \bar x&=\frac{22779}{186}=122.467741\ldots && \text{mean of available values}\\ S_{xx}&=2856003-\frac{22779^2}{186}=66310.30645\ldots && \text{centred sum of squares}\\ s&=\sqrt{\frac{66310.30645\ldots}{185}}=18.932355\ldots && \text{sample divisor }n-1 \end{aligned}

Why this step? Using 200200 would pretend that the missing values were observed. Using 186186 inside the final square root would select the wrong MEI SD convention.

Answer. Mean 122.468 mmHg122.468\,\mathrm{mmHg}, sample SD 18.932 mmHg18.932\,\mathrm{mmHg}.

Check. On a compact 186186-row table, =COUNT(C2:C187)\texttt{=COUNT(C2:C187)} gives 186186; =AVERAGE(C2:C187)\texttt{=AVERAGE(C2:C187)} and =STDEV.S(C2:C187)\texttt{=STDEV.S(C2:C187)} reproduce these values.

Change only the blood-pressure variable to Diastolic. Its own data quality determines its selected count; do not carry over the earlier count automatically.

Worked example 4: a variable-specific cleaning decision

Question. For Age–Diastolic, the notes identify an erroneous zero Diastolic value at Data row 4747. After marking that value unavailable and removing missing required pairs, 185185 observations give ∑x=12768\sum x=12768, ∑x2=908664\sum x^2=908664. Determine their mean and sample SD to 33 decimal places.

Plan. Use the cleaned count and its sample divisor; keep the original workbook and the decision record.

xˉ=12768185=69.016216…mean of the cleaned valuesSxx=908664−127682185=27464.95135…use the same selected sets=27464.95135…184=12.217448…new sample divisor\begin{aligned} \bar x&=\frac{12768}{185}=69.016216\ldots && \text{mean of the cleaned values}\\ S_{xx}&=908664-\frac{12768^2}{185}=27464.95135\ldots && \text{use the same selected set}\\ s&=\sqrt{\frac{27464.95135\ldots}{184}}=12.217448\ldots && \text{new sample divisor} \end{aligned}

Why this step? The erroneous Diastolic value is a problem for this analysis; it does not erase the same person's available Age or Systolic measurements from other analyses.

Answer. Mean 69.016 mmHg69.016\,\mathrm{mmHg}, sample SD 12.217 mmHg12.217\,\mathrm{mmHg}.

Check. Copy the 185185 selected observations to their own range and check that its count, mean and sample SD agree. Do not average the whole column after just hiding the rejected rows.

The original first complete Age–Systolic pairs are useful spot checks on copying:

Original Data rowAge, yearsSystolic, mmHg
223434108108
444848139139
556161137137
666868118118
772828122122
883737153153

Record your variables, excluded codes or errors, final count and units. A tidy table does not establish representativeness: missingness may relate to health or measurement difficulty.

Exam tip

When explaining cleaning for a paired calculation, refer to complete rows for the required variables. Deleting isolated cells breaks the pairs; deleting every row with any missing field discards usable information.

For a fresh summary, keep Age and change the measured variable to Pulse.

Software graphs and outputs explore relationships

An XY scatter diagram puts one person at each complete Age–Systolic pair, so the horizontal and vertical coordinates must stay together.

Select the two columns in the compact 186186-pair table and insert an XY scatter chart, with Age horizontally and Systolic vertically. Label years and mmHg. Do not use a line chart joining people in workbook order: that order is not elapsed time. A point can coincide with another person's point, so fewer visible dots need not mean fewer observations.

Age and supplied systolic average for all 186 complete LDS9 pairs

The pattern suggests positive association with considerable variation. On the compact table, =CORREL(B2:B187,C2:C187)\texttt{=CORREL(B2:B187,C2:C187)} gives approximately 0.6160.616. This describes linear association in these pairs. It is neither a slope nor a claim that 61.6%61.6\% of people have a particular condition. Calculating this coefficient by hand is unnecessary here.

For group comparisons, create separate Sex-group scatter plots with common scales. The available female Systolic values have count 9494, mean 123.798123.798 and sample SD 22.508 mmHg22.508\,\mathrm{mmHg}; the male values have count 9292, mean 121.109121.109 and sample SD 14.397 mmHg14.397\,\mathrm{mmHg}. The observed female spread is greater, but differences in age composition and missingness may affect the comparison. Use a box plot for median and IQR, a histogram for shape, and a bar chart for categorical counts such as Marital.

Empirical probability model

An empirical probability model uses observed frequencies to assign probabilities under an explicitly stated selection rule. Selecting uniformly from the 186186 available Systolic observations gives every one of those people equal probability; it does not select a present-day UK adult.

Choose a numerical threshold as a mathematical event, without turning it into a diagnosis. Count the event and its eligible total using the same cleaned range. The frequency-over-total reasoning is not printed as a separate formula in the formula sheet: learn it.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account