I’m a numbers guy. Growing up, I loved tracking baseball players’ statistics, and I was fascinated by election results. For those who follow my Twitter feed (@hotlinejosh), it’s filled with polling crosstabs and tidbits that shed some light on the trajectory of the presidential race. So it may come as a surprise to hear that I’m growing sick of political polling. Or at least sick of the proliferation of polling, which has introduced a whole lot of noise and precious little clarity to understanding politics.
More than anything else, polling drives political coverage in the media, especially when it comes to the presidential race. More than ever, the polls are provoking pundits to come up with misleading narratives to explain what’s happening. Combine many reporters’ distaste for math with a boatload of unreliable numbers, and you’ve got a recipe for disaster.
Look at the coverage of the presidential race last week. Just as a narrative was taking hold that President Obama was becoming a favorite for reelection, thanks to the improving economy, two national polls (ABC News/Washington Post and CBS News/New York Times) showed that Obama had hit a yearlong low in his job-approval rating. But before that new story line could sink in, Gallup showed his approval rating ticking up to 49 percent, and the Pew Research Center released numbers suggesting that more voters approved of his performance than at any time since last May. (The Allstate/National Journal Heartland Monitor poll also showed Obama above the magic 50 percent mark.) Pundits on the right cherry-picked the gloomy numbers and blamed them on rising gasoline prices, while pundits on the left proclaimed he was in solid shape for reelection. And most observers were left with their heads spinning.
In last week’s Southern presidential primaries, every public poll showed Rick Santorum in third place in Alabama and Mississippi, running behind Mitt Romney and Newt Gingrich. Upsetting the conventional wisdom, Santorum won both those states, while Romney brought up the rear.
It’s not just the presidential contest. The narrative of the Senate campaign in Massachusetts was reshaped this month after several polls, conducted by firms with spotty track records, suddenly showed Sen. Scott Brown, R-Mass., leading Democrat Elizabeth Warren. A previous wave of polling had shown her ahead, and there wasn’t a whole lot of reason for this sudden shift, because the campaign has barely begun.
Some conservatives credited Brown’s opposition to the administration’s contraceptive mandate for religious institutions as a driving force behind his comeback. Others thought that both sides’ agreement to restrict outside spending could have given Brown a useful respite from all the outside attack ads. And just as the media declared Brown the candidate to beat in the race, the Democratic robo-pollster Public Policy Polling released a survey on Tuesday showing Warren actually ahead of Brown. Go figure.
The reality is: We’re just guessing.
Driving much of the polling confusion is the emergence of robo-polling, which has burst onto the political scene like gangbusters over the past decade. These polls, which don’t use a live operator to ask questions, are cheap to commission, and the better ones can produce reliable results close to an election. But the problem with them is that, while they’ve become a quick way to detect movement in a race, they aren’t good at explaining voter preferences. That’s why most professional campaigns pony up big bucks to hire live-caller pollsters because they want to test messages and game out strategy rather than simply capture where the race stands at a particular moment.
Voters responding to automated calls tend to offer more negative feedback than if they had been contacted by live-caller pollsters, perhaps because they’re annoyed at being bothered by a robo-call. When respondents are unfamiliar with candidates, they tend to give them poor reviews—rather than simply say they’re undecided. To wit: In the GOP-leaning state of North Carolina, Public Policy Polling found Romney’s unfavorable rating to be a whopping 56 percent, with just 31 percent viewing him favorably. That doesn’t pass the smell test.
That’s the main reason that recent PPP surveys of Mississippi and Alabama Republicans, asking respondents about Obama’s religion, should be taken with a huge grain of salt. Never mind the fact that the pollster didn’t ask the same question in other non-Southern states, to at least serve as a baseline. PPP was looking to stir up a story line—and it did. More than half of Mississippi voters and nearly half of Alabama voters said they thought Obama was Muslim, a narrative that drew ample coverage in the media, including in the lead segment on NBC’s Today show. These polls were never designed to be trivia contests.
At a time when polls are proliferating, understanding the fundamentals of individual contests is as important as ever. The best way to project the results of the Republican primary contests has been to look at the demographics of the voters: The more-conservative and religious states have backed Santorum and Gingrich, while the more-moderate states have backed Romney. And as compelling as many of the Senate polls are, none of the public surveys have much value until the campaigns start in earnest. Better to look at the strengths and flaws of the individual candidates to project what’s going to happen in the future.
That’s my logical argument for giving less weight to polls. But we are political junkies. For better or for worse, we’ll keep coming back for more.
This article appears in the March 21, 2012, edition of National Journal Daily.