Skip Navigation

Close and don't show again.

Your browser is out of date.

You may not get the full experience here on National Journal.

Please upgrade your browser to any of the following supported browsers:

For Pollsters, One Last Chance To Second-Guess For Pollsters, One Last Chance To Second-Guess

This ad will end in seconds
Close X

Want access to this content? Learn More »

Forget Your Password?

Don't have an account? Register »

Reveal Navigation



For Pollsters, One Last Chance To Second-Guess

New Wild Cards Should Have Minimal Effect As Game-Changers, But Pollsters Still Ponder What They Might Have Gotten Wrong

With less than 24 hours to go before Election Day voting gets under way, the final round of national and statewide polls continues to look very strong for Barack Obama.

Every national poll shows Obama leading, most by margins of 5 percentage points or more, and he has comfortable leads in more than enough statewide polls to project him to win 270 electoral votes. Even in Pennsylvania, where a late push by the McCain campaign has narrowed the gap significantly, the final round of surveys shows Obama leading John McCain by margins of 6 to 8 points.


So here is the question worth pondering one last time, the same question that has been keeping pollsters up late nights all fall: Could the polls be wrong?

Over the last three weeks, I have tried to review the available evidence on three fronts and found relatively little cause for alarm (or for hope, depending on your point of view):

Will the growing number of voters reachable only by cell phone make polls less accurate? If they do, it will be in Obama's favor in polls that are not interviewing by cell phone. National pollsters, such as the Pew Research Center and Gallup, have reported a slight 1-2 point increase in Obama's margin when they include interviews conducted by cell phone.


The inclusion of cell phone interviews may not be the only explanation, but our trend estimates on show Obama leading by a wider margin on national surveys that interview "cell phone only" voters via cell phone (+9.2 as of this writing) than those that do not (+5.4).

Are likely voter models missing a surge of new voters? Probably not, since most surveys use screens or models that will capture new registrants or newly energized voters if they express strong interest and enthusiasm in voting. Even then, Obama is leading by wide margins even on the more traditionally restrictive likely voter models (such as those used by Gallup, Newsweek and the Pew Research Center) that include measures of past voting in their models.

What about the so-called "Bradley Effect?" Will the undecided vote break decisively to McCain, as it did for many white Republicans running against African-American Democrats in the 1980s and early 1990s? Charles Franklin and I looked hard for current evidence of either a race-of-interviewer effect (which was present in some of those races 20 years ago) or a hidden McCain vote among currently undecided voters and found none. The final survey by the Pew Research Center did a similar sort of analysis and found a slight "break" of the remaining undecided vote to McCain, but not enough to make much dent in Obama's lead: It allocated 4 of the 7 undecided percentage points to McCain, 3 to Obama.

We do see some hints of a possible break of undecided voters to McCain in a few battlegrounds. In Pennsylvania, most of the recent movement has represented a shift from undecided to McCain. In Ohio, we notice consistently closer margins on automated telephone surveys than on surveys conducted with a live interviewer, and again the difference looks like a shift to McCain from undecided. Still, Obama is at or above 50 percent in both states and could still carry both even with a decisive "break" of undecideds to McCain.


Taken together, these potential pitfalls offer little hope to McCain voters. But there is one remaining theoretical possibility that amounts to the "nightmare scenario" for pollsters. What if those who refused to be interviewed have very different political views than those who agreed to participate?

This concern is far from trivial, as even the most rigorous national surveys struggle to achieve response rates over 30 percent. And it is hard to know much for certain about those who do not respond because, obviously, we cannot interview them.

One way that pollsters can study the potential impact of non-response is to look at the most difficult-to-interview respondents. In 1997, the Pew Research Center conducted an experiment that involved an unusually rigorous attempt to "convert" to respondents those who initially refused to participate in the survey. They found "strikingly different views" among white respondents "on several race-related questions, with reluctant respondents significantly less sympathetic than amenable respondents toward African-Americans."

Pew could not replicate those findings in a follow-up study conducted in 2003. Still, Andrew Kohut, president of the Pew Research Center, says Pew has always had a harder time completing interviews with a cohort of older, white, less well-educated respondents who typically demonstrate less tolerance on race-related questions. Kohut told me that they identify a demographic cohort that should be 29 percent of adults, but typically represents 20 to 25 percent of adults in their unweighted samples. While they always weight this group up to its appropriate level, he could not rule out the possibility that the missing respondents may be those with less racial tolerance, as they were in Pew's 1997 study.

Although Kohut remains uncertain about how much of problem these harder-to-reach respondents might be, he considers it something "small to worry about" that might mean at most a percentage point or two in the results, not a game-changer for a candidate with a lead as large as Obama's.

Hopefully, we learned from this year's New Hampshire primary to never say never about the potential for a polling failure. Still, all signs from the polls point to an Obama victory.

comments powered by Disqus