Were the 2016 polls better than you think?
WASHINGTON — Voters in key swing states who settled on a candidate just before Election Day, along with polls that surveyed too few voters without a college degree, were among the culprits behind overconfidence that Hillary Clinton was sure to beat Donald Trump in the race for the White House, a new report suggests.
The analysis of 2016's pre-election polls, released Thursday by the American Association for Public Opinion Research, found evidence that Trump voters were more likely than Clinton voters to reveal their preference to pollsters only after the election. But it found little evidence that that's because they were reluctant to reveal their decision to pollsters at all.
The report's authors include widely respected survey researchers from a variety of backgrounds.
They note that state polls missed by a wider margin on average than they had in the four previous presidential elections. Polls showing Clinton ahead in Pennsylvania, Michigan and Wisconsin fueled perceptions that she had a "Blue Wall" in the Electoral College — a wall that collapsed on Election Day.
Although the report faulted the accuracy of state polling, it noted that national poll averages in the final two weeks of the campaign were among the most accurate in the last eight decades at estimating the national popular vote.
At an event hosted by the association, the report's authors said that among the lessons of Trump's surprise victory is the need for survey researchers to highlight the uncertainty inherent in measuring public opinion.
"In the context of an election it's not a bad thing to say 'well, you know, despite this poll it's possible something different could happen,'" said Mark Blumenthal, head of election polling at SurveyMonkey and one of the report's authors.
Among the factors the report suggests were responsible for the state polling miss:
Late deciders
Several states important to Trump's victory had high proportions of voters who said they only settled on the Republican in the last days of the campaign, the report's authors said.
"This was an election where we had two historically unpopular nominees and there's evidence that a sizable share of voters waited until approximately that final week before making up their minds about who they were voting for," said Courtney Kennedy, director of survey research at the Pew Research Center. "And so polls conducted two weeks, three weeks out from the election were not going to be able to detect that late shift in voter sentiment."
Exit polls conducted for The Associated Press and television networks by Edison Research indicated that late deciders in Michigan, Wisconsin, Pennsylvania and Florida supported Trump by double-digit margins.
The report found mixed evidence that this late wave in favor of Trump was directly tied to FBI Director James Comey's announcement on Oct. 28 that the agency would review new evidence in its probe of Clinton's use of a private email system as secretary of state. Her lead in national polls, researchers found, was already eroding at that time.
Shy Trump voters?
The report also found that about 1 in 10 Trump voters, but only about half as many Clinton voters, said before the election that they were undecided or supporting a different candidate, citing data from a Pew Research Center study in which respondents to pre-election surveys were re-interviewed after the election.
This finding could be consistent with the "shy Trump voter" theory, which says that supporters of the brash Republican were reluctant to admit their intention to vote for him because of social pressure.
But researchers found little other evidence for the theory.
"The committee tested that hypothesis in five different ways, and each test either yielded no evidence whatsoever to support that hypothesis or weak evidence," Kennedy said.
The evidence also fits the more benign conclusion that many of Trump's supporters decided only shortly before they voted.
Education
The report suggests some pollsters may have underrepresented voters without college degrees, who were particularly likely to support Trump and have long been less likely to respond to polls.
Nearly all surveys try to correct for such imbalances using procedures known as weighting, which make sure surveys include the voices of the correct proportion of groups such as men and women, whites and racial minorities, and people in different age groups.
But particularly at the state level, not all pollsters weighted to correct for differing levels of education.
"If you did a poll and you had proportionally too many college graduates and you didn't fix that in your weighting, you almost certainly overstated support for Hillary Clinton," said Kennedy, noting that education was more strongly correlated with vote choice than in previous elections.
Turnout
The report says early analysis suggests the way pollsters decided who was likely to vote may have also contributed to polling errors.
Some pollsters rely heavily on turnout from previous elections, either among individuals or demographic groups, to determine who is likely to vote in an upcoming election. Others rely more on self-reported intention to vote. Predicting how turnout will change compared with previous elections is a fraught task, and early evidence from 2016 suggests that turnout was down in some places among key Democratic voting blocs.
However, the report notes that some of the best estimates of turnout among individual demographic groups, including a U.S. Census survey on voting and registration, are not yet available for 2016.