Opinion Polling and Surveys: Best Practices

► “Man on the Street” can’t accurately reflect voter views ◄

By Reporting Elections

Journalists are often tempted to do quick, informal “man on the street” or “vox pop” surveys to supposedly “take the pulse” of the electorate. They are easy to do and provide colorful content for broadcast, print or the Web.

The problem is these kinds of stories rarely provide accurate information about how majorities of voters really feel — whether we are considering a particular constituency or the nation as a whole.  This is because “vox pop” style reports do not provide a sample representative of all voters. They are limited by place, time and circumstance.

In order to do a survey that truly represents the broad cross-section of voters, journalists must use time-tested techniques.

The information and guidelines that follow are from the 2017 Walter Cronkite School of JournalismMedia Foundation 360 workshops that were conducted in Lahore, Islamabad, and Karachi — as well as the subsequent Political Communication class taught at the Cronkite School in the fall of 2017.

Historically, the highest quality polling organizations in the world have been good at determining voter opinion and preferences at a given point in time, providing insights on what voters think about candidates and policies. In this regard, well-executed polls and surveys, that use widely accepted professional practices have been quite accurate.

(For a brief history of polls, check out this PBS story.)

How Sampling Works

A relatively small random sample of around 1,000 people can give a fairly accurate snapshot of what a larger, national population thinks. The larger the sample size, the lower the margin of error.  But beyond 1,000 the margin of error does not change drastically. That is why many national polls use a sample size of around 1,000 people.

For a sample to be truly representative of a larger population it has to be random — meaning the people in the sample were chosen randomly from a group that contains people with the characteristics of the larger population.  This is why a quick “man on the street” survey does not work.

Random sampling can be difficult in rural areas and where people do not have access to technologies like telephones or computers, but every effort should be made to get a truly randomized, representative sample.

Understanding Margin of Error

A margin of error of plus or minus 3 (+/- 3) is typical for a poll of around 1,000 people. That “plus or minus” is important – it essentially means any of the results can vary three percentage points in either direction.

So, a poll that shows Candidate Smith with 48 percent and Candidate Jones with 46 percent could mean that Candidate Smith’s “real” number could be as high as 51 or as low as 45, and Candidate Jones’s “real” number could be as high as 49 or as low as 43. A poll showing 48 to 46 could actually be 51 to 43.

On top of that, the margin of error for a candidate’s lead in a poll is even greater than the poll’s general margin of error. A good rule of thumb is that margin of error for a lead is twice that of the poll’s general margin of error. In a poll with a general margin of error of plus or minus 3 (+/- 3), a lead in which one could have confidence would be more than 6 percentage points.

Questions to Consider

Sheldon R. Gawiser, Ph.D., and G. Evans Witt, from the U.S. National Council on Public Polls have provided the following guide for journalists (and readers) when conducting your own poll or survey — or when evaluating polls or surveys that others have done.

20 Questions 3rd edition_Web ver_2006 2