What went wrong with polls in 2016? Can we trust them now?
President Trump's victory in 2016 came as a shock to many Americans. National polls showed his opponent, Hillary Clinton, leading the race up until the election. Even the Clinton campaign was confident she would win. All data they were looking at seemed to predict her victory.
Mr. Trump, who thought little of data gathering, ended up carrying several key battleground states, including states Democrats thought were secure. By early Wednesday, November 4, 2016, he had won 289 electoral votes, compared to Clinton's 218, securing him the presidency.
Nearly four years later, that jarring difference between prediction and outcome is still front of mind, leading some to distrust or disregard political polling. Scott Keeter, senior survey adviser at the Pew Research Center, said there was "understandable surprise on the part of people."
"I think the profession did take a wake-up call," he said of the 2016 election.
National polling was accurate — Clinton ultimately won the popular vote — but according to Keeter, polls in states that proved to be pivotal, such as Michigan, Pennsylvania, and Wisconsin, had problems.
Education was paramount
One of the key issues with 2016 polling was that much of it didn't account for a person's education.
An examination of the 2016 electorate by Pew found that Whites with a four-year college degree or more education made up 30% of all validated voters, while White voters who had not completed college made up 44%. Only 38% of college-educated Whites said they voted for Mr. Trump, whereas he won by more than two-to-one (64% to 28%) among white voters who had not completed college.
David Dutwin, chief scientist of survey panel AmeriSpeak at NORC at the University of Chicago, said most 2016 polls were not weighted by education, and therefore missed Mr. Trump's significant support from non-college-educated White males.
Weighting, a standard polling practice, is a process that gives more statistical value to groups pollsters know are being underrepresented in surveys. The American Association for Public Opinion Research, AAPOR, defines weighting as "an attempt to ensure that the sample more accurately reflects the characteristics of the population from which it was drawn and to which an inference will be made," noting that "it does not involve any changes to the actual answers to survey questions."
Pollsters weight for characteristics such as age, race, or gender to make sure that their polling sample matches up with census data for those groups in that area. But census data can only go so far, Dutwin said, and isn't adept at accounting for likely voters.
Pollsters were unprepared for White male non-college educated voters to turn out in droves for Mr. Trump — he beat Clinton in that group by the enormous margin of 72 to 23%. Mr. Trump also won among White, non-college-educated women 62 to 34%.
"In 2016 there was a strong correlation between education and presidential vote in key states. Voters with higher education levels were more likely to support Clinton," according to AAPOR's Evaluation of 2016 Election Polls. It also found that surveys were skewed just by who answered the phone.
"Recent studies are clear that people with more formal education are significantly more likely to participate in surveys than those with less education," according to the evaluation. "Many polls – especially at the state level – did not adjust their weights to correct for the over-representation of college graduates in their surveys, and the result was over-estimation of support for Clinton."
Never before has education level so explicitly swayed an election. "That hurt, whereas it hadn't really hurt in years prior," Dutwin said.
What's changing?
According to J. Miles Coleman, associate editor of Sabato's Crystal Ball at the University of Virginia's Center for Politics, the 2016 election showed that accounting for education is now as important as race and gender for conducting an accurate poll. He said "the biggest lesson, especially in the Trump era, is educational attainment as a predictor of people's voting habits."
Keeter said the polling profession recognizes that education has "become a schism in our contemporary politics," and needs to be represented for accurate polls. "(Pollsters) are not going to let that happen again," he said of the 2020 election.
Polling in battleground states, however, is often left up to small institutions or universities that cannot easily adapt. In its 2016 evaluation, AAPOR wrote that state-level polling is "often under-budgeted."
"Errors in state polls like those observed in 2016 are not uncommon. With shrinking budgets at news outlets to finance polling, there is no reason to believe that this problem is going to fix itself," reads the evaluation. AAPOR recommended "well-resourced survey organizations" to collectively finance "high quality state-level polls so as to reduce the likelihood of another black eye for the profession."
"Unfortunately you kind of have to vet the polls as well," Coleman said.
CBS News has always weighted for education, according to Kabir Khanna, CBS News' senior manager of election analytics. Khanna said races in battleground states began to tighten as the 2016 election came to a close, prompting the CBS News polling unit to envision a scenario where Mr. Trump could win if he took the Rust Belt states — a scenario that then played out.
The unit takes every state into account, even those that aren't considered crucial for victory, to be ready for the unknown. Khanna said the logic is circular: People poll states they think are going to be competitive, which leads to a lack of data on states that are unexpectedly contentious.
"We only know what is going to be competitive if we have the data," he explained.
To prepare for the unknown, CBS News' Battleground Tracker uses data from neighboring states, or states with similar kinds of voters, to create a more accurate picture of states that have less polling.
In 2016, for example, scarce polling in certain states like Michigan or Wisconsin led some to believe they were not as competitive as they turned out to be. That picture might have been improved by considering that these states were full of the same kinds of voters shifting to the Republican Party elsewhere.
Undecided voters
Predicting which candidate undecided voters would turn out for was another critical issue in 2016 polling.
Both Clinton and Mr. Trump were unpopular candidates, creating a large percentage of unpredictable voters close to the election. According to Keeter, about 15% of voters said they still hadn't decided who they would vote for late in the campaign.
Historically, Dutwin explained, undecided voters break close to the electorate as a whole. In 2016, however, Mr. Trump won a strong majority of voters who said they hadn't decided who they were voting for until Election Day.
The share of undecided voters appears to be much smaller in 2020, down to about 4 to 6% in battleground states. And according to Khanna, most voters have been sure of their choice since the summer.
Decisive voters means less volatility on Election Day — but more polarization. "There's been an increasing trend in American politics to vote with your party," Khanna said.
Can Trump pull off another upset?
Coleman described Mr. Trump's 2016 win as a "royal flush." Asked about what we could expect this year, he responded, "In cards you don't draw a royal flush every time."
In addition to increasing party loyalty, the lack of a popular third-party candidate could also create stability in this year's race. According to Coleman, third party candidates took 4 to 5% of the national vote in 2016, a share that is expected to go down this year.
CBS News polling has identified six "toss-up" states, and seven more that are leaning towards one candidate, but could still swing.
Coleman said North Carolina and Arizona are two states he is watching on election night.
Barack Obama narrowly carried North Carolina in 2008, and Democrats have been trying to win it again ever since. "If Biden can carry North Carolina, that's probably a sign that he's doing well across the country," Coleman said. Arizona is another state that Democrats have been eyeing. Phoenix's Maricopa County, which accounts for a majority of the state's total votes, was the largest county in the country to go for Mr. Trump in 2016, but the margin was slim, according to Coleman.
"If Biden can flip Maricopa county, that would be big," he said.
Dutwin, too, is watching North Carolina, as well as Pennsylvania and Florida.
Florida, a familiar toss-up on the East Coast, has already begun counting mail-in ballots, meaning it could produce some of the first results. Dutwin said that if Florida breaks for Biden it will be "very hard to see a victory path for Trump;" but if Mr. Trump takes the state, there are other ways for Biden to win the Electoral College. "(It's) a one-sided canary in the coal mine," he said.
The most unpredictable aspect of the 2020 election may be the coronavirus. Keeter called it the "wild card" this year.
According to CBS News polling from mid-October, people who are very concerned about the coronavirus cite it as a major factor in their vote — ahead of the economy — and they are backing Biden over Mr. Trump.
"We just do not know how well people are going to be able to act on their intention this year," Keeter said.
More people most concerned about the virus say they plan to vote by mail, rather than in person, and most of those who are doing so say they've usually voted in person in the past. Overall, a large proportion of the electorate is planning to vote by mail, or already has, with a distinct partisan split: Democrats prefer voting by mail, Republicans in person.
What remains to be seen is how quickly mail-in ballots will be counted, will there be significant roadblocks that prevents those ballots from being counted, and how many people will show up on Election Day itself.
"The big question is turnout," Khanna said. Adding, "there aren't many people left to persuade."
