02 Section A, Key question 3 - Questionnaires and structured interviews (part 1)

Two ways to administer a standardised survey

A survey can give people the same questions while allowing them to complete the answers themselves or asking an interviewer to record them.

A questionnaire is a set of questions used to collect information from respondents, the people answering the research questions. In the self-completion approach, respondents read the questions and enter their own answers. They might complete a paper form posted to their home, an online form or a form handed out in a community centre. A questionnaire is also the instrument an interviewer can use: the key comparison here is between self-completion and interviewer administration.

Structured interviews can take place face to face or over the telephone. The interviewer may read fixed response options, show a response card or record an answer in the respondent's own words. Standardisation means keeping the procedure consistent, including agreed instructions and any permitted clarifications. A prearranged rule may skip an employment question for someone who is not employed; this is still standardised if everyone meeting that condition follows the same route.

Consider an original study of access to promotion. A self-completion form asks workers how many promotions they applied for in the previous year. A structured interviewer reads the same question to each worker and records the number. Both can compare workers using a common measure. In the interview, however, another person is present and can encourage completion or influence the answer. In self-completion, the respondent has more control over pace but cannot usually ask a trained interviewer what a confusing term means.

Standardisation supports reliability, the consistency of a measure or procedure. It reduces differences caused by one interviewer asking about a calendar year and another asking about a worker's entire career. It does not ensure that respondents tell the truth or interpret the question identically. Nor does reliability mean that people's income or attitudes must remain unchanged when they are genuinely changing.

Question format and the evidence produced

The format of a question shapes what respondents can say and how their answers can be analysed.

A closed question provides predetermined answer categories, such as whether a worker received a promotion: yes, no, or do not know. Quantitative data express amounts or distributions numerically. Counting the yes and no responses makes it possible to compare the proportion promoted in different groups. The categories themselves need not be numbers for their frequencies to be quantitative.

An open question allows respondents to formulate their own answer. “Describe what made it difficult for you to apply for promotion” might reveal worries about childcare, informal recruitment or a manager's assumptions. Qualitative data provide accounts of meanings, experiences and processes, commonly in words. They can illuminate how an inequality is experienced rather than just how often a fixed category is selected.

Open questions can occur in a self-completion questionnaire or in a structured interview. An interviewer might ask everyone the same open question and record the replies verbatim, meaning in the words used. The presence of an open question does not automatically make the interview unstructured. Equally, an interview does not automatically produce qualitative data: a schedule of yes/no and numeric questions can produce a quantitative dataset.

Question format and data type are related but not identical. A field allowing someone to enter their own number of caring hours has no fixed list of categories, yet the answer is quantitative. An open request for an account of how caring affects their day can instead produce qualitative evidence.

The research purpose determines which is useful. If the aim is to compare reported promotion rates, consistent closed categories simplify counting. If the aim is to understand why workers think they were excluded, a free response may reveal explanations the researcher did not anticipate. Yet a small answer box can restrict an open reply, and a tick box cannot explain a complex experience by itself.

Researchers can code open answers by assigning them to categories. For example, accounts mentioning the cost of nursery places might be coded as “childcare cost”. Counting that category then adds a quantitative summary. The original accounts remain qualitative evidence; the coding decisions should be stated because grouping answers can hide differences. Someone who cannot afford care and someone who cannot find care have related but different difficulties.

Writing clear and neutral questions

Useful survey questions ask respondents about a clear idea without steering them towards the researcher's preferred answer.

A leading question suggests which answer is expected. In this original example, “Do you agree that unfair employers always deny mothers promotion?” supplies a judgement, generalises to every employer and invites agreement. If the aim is to study workers' own promotion experiences, a neutral alternative is “During the past 12 months, have you applied for a promotion at your workplace?” followed, where relevant, by questions about the outcome and the worker's explanation. This separates an observable report from an interpretation. A worker's belief that an outcome involved discrimination is important evidence about their experience, but the answer alone does not establish the employer's motive.

Questions should also avoid being double-barrelled, meaning that they ask about two separate issues while allowing only one answer. “Are you satisfied with your pay and working hours?” cannot express satisfaction with pay but dissatisfaction with hours. Split it into two questions, each with its own response scale. For a satisfaction question, a balanced scale gives comparable positive and negative choices, such as very dissatisfied, dissatisfied, neither, satisfied and very satisfied. Allow a suitable “do not know” or “not applicable” option where respondents cannot judge. More questions can therefore produce clearer evidence than one apparently efficient question.

Vague language can create a less obvious problem. “Do you earn a good income?” measures a judgement that may vary with household needs and comparison groups. To measure pay, define what counts: individual or household income, before or after deductions, and the time period. For an original employee survey, “What was your gross pay from this job last calendar month, before tax and other deductions?” is clearer. Explain “gross pay” rather than relying on specialist language, and test whether employees with irregular hours can answer. This question measures pay from one job; it is not a complete measure of household resources.

Validity concerns whether the evidence captures what the research claims to investigate. Precise wording can improve validity by reducing ambiguity. But an accurately reported salary is still insufficient to measure every dimension of social class or economic insecurity. Measurement needs to fit the concept as well as be understandable.

The ONS Question Design Guidelines emphasise a clear target population and intended measure, simple language, and consideration of sensitivity. A pilot is a small trial before the main study. Asking trial respondents to explain what they understood can expose ambiguities; merely checking that the form opens successfully would miss them.

Response categories, order and unpaid care

A clear question can still produce misleading answers if its response categories do not fit people's circumstances.

Closed categories should be mutually exclusive, so a respondent does not fit two alternatives when only one is allowed, and sufficiently exhaustive, so the relevant possibilities are covered. In an original unpaid-care survey using whole-hour answers, categories of “0–10”, “10–20” and “20 or more” overlap at 10 and 20. A coherent revision is “0”, “1–9”, “10–19”, “20–34”, “35–49” and “50 or more hours”. Zero is distinct from a missing response. A category such as “prefer not to say” records refusal, not absence of care.