Real Polls Vs. Push Polls

By Kathy Frankovic, CBS News director of surveys

Romney's specific religion could be even more of a factor. The CNN/Opinion Research Corporation Poll conducted last month found 19 percent -- one in five Americans -- saying they'd personally be less likely to vote for a presidential candidate who was Mormon (only 3 percent would be more likely). In a June CBS News Poll, twice as many registered voters -- 43 percent -- claimed most people they knew would not vote for a Mormon for president. Republicans were no different from Democrats on this.

Faith as an issue could matter this year. And it's certainly something to poll about. But did the New Hampshire questions about Romney's religion violate some basic rules of polling? Was it a "push poll?"

Push polls really aren't polls at all. But charges that opponents are conducting them make great headlines in campaigns. In 2000, a 14-year old boy reported that an interviewer for George W. Bush said Bush's GOP presidential rival John McCain was a "cheat" and a "liar." McCain promptly accused Bush of conducting "push polls" in South Carolina, and won headlines and sympathy. Even politicians knew then and know now that "push polls" are wrong.

A push poll is political telemarketing masquerading as a poll. No one is really collecting information. No one will analyze the data. A push poll is very short, even too short. (It has to be very short to reach tens of thousands of potential voters, one by one). It does not include any demographic questions. And, of course, a push poll will contain negative information -- sometimes truthful, sometimes not -- about an opponent.

Push polls mislead the public, and not just about an opponent. They mislead the public about polls: Callers claim they are conducting a poll when all they are doing is spreading negative information.

But sometimes charges of push polling are themselves misleading. Not all questions that seem negative are part of push polls. Candidate organizations sometimes conduct polls with questions that contain negative information about opposing candidates. These polls, which are not push polls, are conducted for the same reasons market and advertising researchers do their work: to see what kinds of themes and packages move the public.

Advertisers want to figure out the best way to reach buyers; candidate pollsters need to motivate voters. These real polls are full-length, covering more topics than just some negative questions about an opponent. They include demographic questions that allow researchers to categorize respondents.

For example, the 2000 Bush campaign poll in South Carolina that was accused of being a "push poll" was a 20-minute long questionnaire. According to the Bush campaign, there were only 300 interviews. While that sample size is low for a media poll, it is standard in many campaigns and nowhere near the number of calls a push poller would make. No one would try to reach tens of thousands of potential voters with a push poll lasting 20 minutes. This year's New Hampshire anti-Romney poll also seemed to take about 20 minutes, far too long for a push poll.

But we are left with something of a poll mystery. No one is quite sure who is responsible. Mark Blumenthal at pollster.com has tried to track it down, and Mark Hemingway's article at National Review Online suggesting that the poll may have been conducted by Romney's own campaign searching for potential problems has provoked responses -- and denials -- from campaigns and campaign workers.

The American Association for Public Opinion Research (a national association of survey researchers which tries to uphold standards) issued a press release to make the distinction between a real poll and a push poll. "Negative or disturbing information about a candidate does not automatically make a survey a push poll," said AAPOR president Nancy Mathiowetz. "Message testing, when campaigns test the effectiveness of possible messages about opponents and even themselves, is very different; and it is a legitimate form of surveying."

There are ethical questions that can be raised about the anti-Romney poll, but they are questions that should be raised about every poll. Are the facts as stated correct? AAPOR once censured a pollster because he got the facts wrong in his questionnaire. Are respondents being misled? Are the questions themselves harmful to respondents?

Sometimes real polls do include questions with negative information about one candidate or another. But what happened in New Hampshire probably wasn't a push poll. But it's become a great campaign story!
By Kathy Frankovic