Survey figures show up in almost every type of news story. And they are misused more than any other data type in everyday journalism.
As a beginner editor, knowing what to question about a survey will make your work noticeably more reliable.
Start with sample size and method
A survey of 312 people conducted online is different from a nationally representative survey of 4,800 respondents with quota sampling. Both might appear in a news feed portal under the same confident headline.
Your first question is always: who was surveyed and how were they selected?
What to check in the article text
- Does the writer say the survey was conducted among a general population when it was actually a self-selected online panel?
- Is the margin of error mentioned? If the difference between two groups is smaller than the margin of error, the comparison is not meaningful.
- Has the writer used the word majority for anything above 51 per cent without noting how close the split was?
- Is the survey recent, or is a two-year-old figure being used to describe current behaviour?
The question wording problem
Survey results depend heavily on how questions were asked. If the article does not link to the original questionnaire, that is worth flagging to the writer.
A question like would you support X if it meant Y produces very different results than would you support X alone. The phrasing shapes the data.
A note on commissioned surveys
If a brand or lobby group commissioned the survey, that needs to be stated in the article. Readers deserve to know who paid for the research they are reading about.
Editing survey data is less about maths and more about asking the right questions before anything goes to publication.