Sb11, Sb12 - Regression on a non-random variable

Sb11, Sb12 - Regression on a non-random variable

Fit and use a least squares line when an experimenter controls the input and measures a variable response. Calculate residuals, judge a linear model informally and keep predictions within the role and range the model supports.

A controlled input and a random response

How can a best-fit line miss every observed point yet pass exactly through the point formed from the two means?

The line balances the observations as a group. It is not an instruction to join measured points, and it need not contain any individual observation.

You already know

Plot a controlled independent variable horizontally and its measured response vertically. Random-on-non-random data differ from two measurements that both vary across randomly sampled individuals.

Suppose an experimenter fixes a sensor's input level xx. Repeated measurements at that level can give different responses YY, owing to measurement variation or other uncontrolled influences. The input itself is treated as known, with negligible measurement error.

Regression of a response on a controlled input

A regression model for random YY on non-random XX uses fixed input values to estimate the underlying “true” response. Here that response is modelled as a linear function of the input; observations vary around it.

On a scatter diagram, repeated measurements at the same input lie on the same vertical line. Calculating the fitted line does not require the Pearson population-correlation test, which is inappropriate for this controlled-input setting.

Key idea

The model predicts the response at a chosen input. The controlled input is not another random response to be predicted by reversing the line.

Identify the model's direction.

Finding the least squares line

The least squares line minimises the sum of squared vertical differences between observed responses and fitted values. Large vertical discrepancies therefore count more than small ones. You need to use the fitted-line formula, not derive it.

Least squares regression of y on x

Write the fitted value as y^=a+bx\widehat y=a+bx. Then

b=SxySxx,a=yˉ−bxˉb=\frac{S_{xy}}{S_{xx}},\qquad a=\bar y-b\bar x

Equivalently,

y^−yˉ=b(x−xˉ)\widehat y-\bar y=b(x-\bar x)

The centred sums are

Sxx=∑xi2−(∑xi)2n,Sxy=∑xiyi−(∑xi)(∑yi)nS_{xx}=\sum x_i^2-\frac{(\sum x_i)^2}{n}, \qquad S_{xy}=\sum x_iy_i-\frac{(\sum x_i)(\sum y_i)}n

The input values must not all be identical.

In the formula booklet.

For raw data, a paired-variable calculator fit is expected: identify its coefficient convention rather than assuming that every calculator uses the same letters for intercept and gradient. For summary statistics, show your substitution.

Worked example 1: a calculator fit checked from totals

Question. Four fixed input levels and measured sensor responses, in appropriate units, are:

xx00112233
yy11222255

Find the least squares line for response on input.

Plan. Fit the paired data, then check gradient and intercept with the centred sums and means.

xˉ=64=1.5,yˉ=104=2.5column meansSxx=14−624=5input variationSxy=21−6(10)4=6paired variationb=65=1.2gradienta=2.5−1.2(1.5)=0.7intercept\begin{aligned} \bar x&=\frac64=1.5,\quad \bar y=\frac{10}4=2.5&&\text{column means}\\ S_{xx}&=14-\frac{6^2}{4}=5&&\text{input variation}\\ S_{xy}&=21-\frac{6(10)}4=6&&\text{paired variation}\\ b&=\frac65=1.2&&\text{gradient}\\ a&=2.5-1.2(1.5)=0.7&&\text{intercept} \end{aligned}

Why this step? We minimise vertical response errors, so the denominator is the input sum SxxS_{xx}, not a response sum.

Answer. y^=0.7+1.2x\widehat y=0.7+1.2x.

Check. At xˉ=1.5\bar x=1.5, the line gives 0.7+1.2(1.5)=2.5=yˉ0.7+1.2(1.5)=2.5=\bar y.

Change only the last response, from 55 to 44, and express the changed data through summaries.

Worked example 2: one observation changes

Question. The changed data have

n=4,∑x=6,∑y=9,∑x2=14,∑xy=18n=4,\quad \sum x=6,\quad \sum y=9,\quad \sum x^2=14,\quad \sum xy=18

Find the new regression line of response on input.

Plan. Recalculate the response mean and cross-product sum, retaining the unchanged input variation.

xˉ=1.5,yˉ=94=2.25new response meanSxx=14−364=5unchanged inputsSxy=18−6(9)4=4.5new paired variationb=4.55=0.9new gradienta=2.25−0.9(1.5)=0.9new intercept\begin{aligned} \bar x&=1.5,\quad \bar y=\frac94=2.25&&\text{new response mean}\\ S_{xx}&=14-\frac{36}4=5&&\text{unchanged inputs}\\ S_{xy}&=18-\frac{6(9)}4=4.5&&\text{new paired variation}\\ b&=\frac{4.5}{5}=0.9&&\text{new gradient}\\ a&=2.25-0.9(1.5)=0.9&&\text{new intercept} \end{aligned}

Why this step? Changing one response affects both its mean and its cross-products. Keeping the old intercept would leave the new line missing the new mean point.

Answer. y^=0.9+0.9x\widehat y=0.9+0.9x.

Check. At x=1.5x=1.5, the predicted value is 2.252.25, the new response mean.

The opening puzzle now has its exact answer. The original line misses all four observed points but passes through (1.5,2.5)(1.5,2.5). Every least squares line with an intercept passes through its mean point.

[DIAGRAM: asset_name: A fitted line passes through the means; asset_slug: controlled_regression_mean; caption: The original fitted line misses all four observations but passes through their mean point; description: The four blue observations (0,1), (1,2), (2,2), (3,5) are plotted with input x horizontally and response y vertically. The blue line y equals 0.7 plus 1.2x passes through a sage mean point at (1.5,2.5). Short vertical grey segments show the differences between each observation and the line. The mean point is labelled separately and no horizontal-distance minimisation is implied.]
The original fitted line misses all four observations but passes through their mean point

Watch out

Wrong: reuse nn or totals from observations that were removed or corrected. Right: make sure every summary describes the same final set of pairs before fitting.

Change the last response once more.

Estimating in the appropriate direction and range

A fitted response is an estimate of the underlying response at an input, not a promise that the next observation will equal it.

Interpolation and extrapolation

Interpolation uses an input within the observed input range. Extrapolation uses one outside it. Extrapolation is more doubtful because the fitted linear relationship may not continue beyond the observed conditions.

Continue this lesson.

You’re previewing NovaLearn without an account. Create a free account to read the rest.

Create free account