NEW IPO Logo - by Charles Larry Home Search Browse About IPO Staff Links

The Pulse                      

How accurate
are call-in polls?


By NICK PANAGAKIS

NICK PANAGAKIS

Broadcasters in increasing numbers are using call-in polls as a means of determining public opinion on current issues. By simply picking up their telephones, audiences register opinions about sending U.S. troops to the Middle East, whether abortion should be legal, even guilt or innocence in a murder trial.

Polling experts have registered their opinion about call-in polls, too. Branding call-in polls as "pseudo polls," they point to likely result bias due to station audience demographics and the cost associated with participation. But more importantly, because the "sample" selects itself, one side of an issue may be more motivated to respond to call-in polls than the other.

News directors have responded with a caveat when reporting call-in poll results: that they were based on unscientific samples. Pollsters say this disclaimer is insufficient because the public is likely to be deceived by the large number of call-in responses. Moreover, pollsters say stations would not air a news story when an unreliable source is the only attribution.

On the evening of January 28, results from a CBS News call-in poll helped to resolve the controversy. Immediately following the State of the Union address by President Bush, an hour-long special show invited viewers to call an 800 number to register their views about the economy. At the same time, a random sample of 1,241 adults who had previously been recruited to watch the show, called CBS to answer the same questions. This random sample was weighted (as many samples are) to correct for any bias due to selection, demographics and nonresponse.

A total of 317,500 viewers were successful in phoning the 800 number to register their views. The Chicago Tribune reported that 24.6 million adults tried to call, which was the largest such response in history and caused one CBS news executive to be "very pleased." To broadcasters the level of audience participation — not reliability — appears to be the measure of success for call-in polls. Pollsters have always argued that on questions of sample and sample size how is more important to poll reliability than how many.

Call-in poll results were compared on the air with the scientifically selected

Financial situation,
now v. four years ago

Random
sample

Call-in
poll

Difference

Same

44%

17%

- 27

Worse

32%

54%

+ 22

Better

24%

29%

+ 5

Worried about you/family member
losing a job

Random
sample

Call-in
poll

Difference

Yes

48%

64%

+ 16

No

52%

36%

- 16

Whether media exaggerates
economic conditions

Random
sample

Call-in
poll

Difference

Yes

35%

39%

+ 4

No

65%

61%

- 4

Source: CBS News.


30/March 1992/Illinois Issues


sample results. We use those results as experimental evidence to prove or disprove the reliability of call-in polls (see the box).

• To the question asking about their personal financial situation now versus four years ago, 54 percent of call-in poll callers said they were worse off, which was 22 percentage points higher than in the random sample. Understandably, people who are worse off are far more likely to reach for their phones to register a complaint about the economy than people with no change in their economic condition.

• When asked if they were worried about the possibility of a job loss in their family in the coming year, 64 percent of those responding to the call-in poll said "yes," which was 16 percentage points higher than in the random sample. Once again, people concerned about a family job loss were far more motivated to respond to the call-in poll about the economy than those who felt secure in their jobs.

Concern about the economy, the subject of the CBS show, is what prompted viewers to reach for their phones. If differences are not as great between the random poll and the call-in poll on questions unrelated to economic concerns, the case becomes even stronger that bias or distortion occurs in the call-in poll.

A question regarding media coverage of economic conditions provides this evidence from the CBS program. This unexpected question resulted in smaller differences between random sample and call-in results. If the subject of the call-in poll had been billed as media coverage, the difference would have been greater on the media question (with call-in respondents more negative), and economic question results would have become more similiar.

News directors will probably continue to use call-in polls and to impress both their audiences and their managements with the large numbers who respond. The solution is clear. Broadcasters must now adopt a new caveat for call-in polls: Based on an experiment conducted by CBS News, results can be expected to differ from a scientific sample by as much as plus or minus 27 percentage points!•

Nick Panagakis is president of Market Shares Corporation, a marketing and public opinion research firm headquartered in Mount Prospect. Panagakis, a member of the National Council on Public Polls, is best known for preelection and exit polls conducted for the news media in Illinois, Missouri and Wisconsin.

March 1992/Illinois Issues/31


|Home| |Search| |Back to Periodicals Available| |Table of Contents||Back to Illinois Issues 1992|
Illinois Periodicals Online (IPO) is a digital imaging project at the Northern Illinois University Libraries funded by the Illinois State Library