Why Polling Doesn't Match Election Results
The results of the most recent CBS News/New York Times Poll have been criticized –- by those who don't like its results. The criticism is that the answers to one question in the poll -- how did you vote in the 2008 election for president -- don't mirror the actual vote.
But that is nothing new. This poll and others have asked about past vote for many years, and the results rarely match the voting results.
Here's the most extreme example: The University of Michigan's National Election studies, arguably the best academic analyses of why Americans vote the way they do, found huge disparities in how Americans reported their 1960 vote in their 1962 and 1964 post-election studies. When Americans were asked just after the 1960 election how they voted, John Kennedy and Richard Nixon were tied in the poll (matching the results: Kennedy just eked out his 1960 victory). Two years later, Americans recalled giving Kennedy a comfortable win. In 1964, after Kennedy was assassinated, the difference was even more dramatic – Americans "remembered" that they had elected Kennedy by a two to one margin!
It goes in both directions – Democratic AND Republican: In January 2002, when George W. Bush's approval rating was 82 percent, Americans recalled that they had given him a 12-point victory over Al Gore in November 2000. If only they had! We would have been spared the 35 days of electoral wrangling that followed that election.
In the latest CBS News/New York Times Poll, Americans claim they gave Barack Obama a 48 percent to 25 percent victory over John McCain (he won by just seven points). Other polls also show an over-reporting of Obama's support: an L.A. Times Poll conducted in December showed a 20 point lead for Obama among voters.
Why does this happen? Two reasons. Many voters simply do not want to admit to a poll interviewer that they didn't vote in the election. Polls regularly show a significant overstatement of voting behavior: an April 2009 Gallup/USA Today poll showed 81 percent of Americans said they voted in the 2008 election. Actual turnout in the election has been estimated at about 57 percent of the voting age population.
Second, voters' memories are affected by their current opinions. If the elected President is particularly popular, the desire to remember voting for him increases. Barack Obama's approval ratings are in the 60s, both in the latest CBS News/New York Times Poll and most other polls. Voters may like to associate themselves with the winner (in their memories, at least), and many are reconciling their current opinions with how they voted last November.
Of course that means that if presidents become unpopular, memories will change, too. In June 2005, voters said they had supported John Kerry over George W. Bush by a point, 38% to 37%. (President Bush won re-election by three points.) President Bush's popularity was in the low 40's – and declining -- at the time. And there were moments during the Watergate scandal in 1974 when voters did not remember that they had elected Richard Nixon in a 20-point landslide (the margin of his victory over George McGovern shrunk along with his sinking approval rating).
Voters may also be affected by the questions they've already been asked in the poll. Topics that tend to favor Democrats, such as health care or the economy, could cause a higher Democratic response to the 2008 vote question, which is asked toward the end of CBS News Polls.
Why ask the question if it yields data that don't reflect the actual voting results? It is useful to pollsters as another way of understanding a president's popularity – and his weaknesses.
So, as most pollsters know, criticizing a poll's results on the basis of how people respond to this particular question makes no sense.
Mark Blumenthal, the editor and publisher of Pollster.com, addresses the issue on his blog here.
And Kathleen Frankovic wrote about respondents lying to pollsters in a 2007 column for CBSNews.com.