In the end, after all the unskewing, the re-weighting and the hand-wringing, the polls were right — for the most part.
For months, pundits and pollsters debated whether public surveys were capturing accurate snapshots of the coming electorate, with conservatives in many cases alleging that polls were sampling too many Democrats and providing an artificial boost to Democratic candidates. These polls, and the averages and statistical models based on them, such as Real Clear Politics, Pollster.com, and The New York Times's blog FiveThirtyEight, drove the spurious narrative that President Obama held a significant advantage in the states most likely to decide the White House race, they argued.
Those critics were wrong, as Tuesday night showed, and the polls were correct: Obama did have that advantage, and it has propelled him to a second term. But the election also showed the polling industry is still facing increasing challenges related to how the various polls are conducted. The demographics of the country are changing, and the way in which Americans communicate with one another is changing even faster; both of these movements are significant threats to the future of telephone polling, whether conducted by live interviewers or automated computers.
Both live-caller and automated robo-polling methodologies had their share of hits and misses at the national and state levels. Nationally, polls from ABC News/Washington Post, NBC News/Wall Street Journal, and Pew Research Center performed well, while Gallup and the automated pollster Rasmussen Reports missed the mark, mostly showing Romney leading by various margins over the final few weeks of the campaign. The latter two are so prolific that Democratic groups commissioned their own set of polls late in the campaign to provide a counterweight to their results.
At the state level, some polls again performed fairly well, while others were less predictive. Overall, in many swing states, while Obama led in the averages produced by websites like Real Clear Politics and The Huffington Post's Pollster, he outperformed those averages.
"If anything, the polls understated" Obama's margin of victory, said Mark Blumenthal, the cofounder and editor of Pollster.
Blumenthal, speaking on Thursday at an event at the National Press Club, added that automated pollsters like Rasmussen Reports and the recently emerged, Florida-based firm Gravis Marketing were "the reason that those estimates" were too pessimistic for Obama. Those firms produced polls that consistently showed Obama underpeforming relative to other polls and the final election results. For example, Rasmussen's final polls showed Obama trailing by 3 percentage points in Colorado, 2 points in Florida and Virginia, and 1 point in Iowa. In Ohio and Wisconsin, the final Rasmussen polls showed Obama tied with Mitt Romney.
In most cases, these automated polls call only landline phones, though some firms, such as Rasmussen, augment their polls with self-selected Internet panels of respondents who only own cell phones as a means of incorporating the more than a third of Americans who are cell-only. The cell-only population tends to be younger, less white and is more likely to rent rather than own their homes, compared to the overall population. Automated polls do not meet National Journal's standards for publication, partly because of the cell phone issue.
As a result of their methodology, automated polls often miss younger, more racially diverse voters; that is particularly true of automated polls conducted in a single day, like those from Rasmussen. Their respondents are disproportionately older and whiter. But turnout did not drop among younger voters as some pollsters had predicted, according to exit polls. And the percentage of the electorate that was white continued to drop, to just 72 percent.
Asked by Slate about why his polls showed a closer race, if not a slight Romney advantage, Scott Rasmussen cited these two factors. "The two differences I noted were share of white vote falling to 72 percent," Rasmussen told Slate. "Also, youth turnout higher and senior turnout lower than expected."
The same phenomenon appeared to develop among some live-caller pollsters, both Republican and nonpartisan. For example, the independent firm Mason-Dixon Polling & Research produced two polls for Florida's two biggest newspapers over the campaign's final month that showed Romney ahead by 7 points and 6 points, respectively. The Miami Herald published a story on Wednesday acknowledging that these polls undersampled young and Hispanic voters.
These polls may have underestimated turnout among these groups because they were less likely to pass through the pollsters' likely-voter screens — batteries of questions that measure respondents' self-reported intention to vote, the attention they are paying to the campaign, their enthusiasm about voting, their voting history and other qualities typically predictive of behavior on Election Day.
"It does seem to me that these likely voter models were showing young people as less enthusiastic," Republican pollster Kristen Soltis Anderson, who has focused on GOP outreach to younger voters, told National Journal. Voters under age 30 voted for Obama by a 23-point margin, according to exit polls, 60 percent to 37 percent.
Democrats did not let these polls stand for themselves. Groups such as Project New America and Americans United for Change commissioned multiple surveys in battleground states over the final weeks of the campaign, in some cases releasing more than one poll in a state per week, in large part as an effort to combat what one prominent Democratic strategist involved in the effort called the "negative storyline" formed by automated polls and other surveys that understated Obama's vote share. These were live-caller polls, conducted using landlines and cell phones, and in most cases they reflected Democrats' view that the demographic composition of the electorate would be more favorable to Obama.
"There was a lot of concern that there was a lot of bad polling out there," said the strategist, who was granted anonymity in order to discuss the project. "Rasmussen floods the zone," the strategist added.
Ethan Axelrod, communications director for Project New America, told National Journal that the polling the group commissioned provided "an accurate picture of where we thought the electorate was." Project New America released multiple polls conducted by the Portland, Ore.-based Democratic firm Grove Insight during the final month of the campaign; in some states, they published results of a handful of polls in a two-week period.
"We felt good about the polls we were doing and thought they should be released to the public," Axelrod said.
Expect this to be the norm over the next few election cycles. More and more, polls are conducted by partisan organizations, or organizations catering to increasingly partisan media outlets.
In previous election years, Democrats relied on their partisan automated polling firm, the Raleigh, N.C.-based Public Policy Polling, to provide a counterweight, but the presence of more expensive, live-caller polling this year was a departure from that approach. It is worth noting that PPP's final preelection polls were among the most accurate of all the outfits polling the campaign.
Democrats won the argument over the composition of the electorate in this election, but that fight is likely to be joined once again in 2014 and 2016, when pollsters for both sides are likely to be armed with competing surveys.
This election was a victory not just for Democrats, but also for the sort of data analysis — through averages and prediction models — that is likely to remain a staple of political coverage for elections to come. Nate Silver, the baseball statistician-turned-New York Times political analyst who used polls and economic data to predict a relatively easy victory for the president, is the most prominent member of this movement, but he is hardly its only devotee.
"Nate has become the personification of an approach," said Blumenthal.
That data-heavy approach seems likely to play a larger role in the coverage of understanding of future elections, raising the stakes for partisans, pollsters, and media to get the data right.