Political Pollsters Still Don’t Know What They’re Doing

Many of the problems behind inaccurate surveys in last year’s presidential election are surfacing again in the midterm races.

TO GO WITH AFP STORY BY JULY FRAYSSE - A woman makess a phone call with an iPhone in Bordeaux on May 3, 2013. 
AFP/Getty Images
Scott Bland
Add to Briefcase
See more stories about...
Scott Bland
Dec. 8, 2013, midnight

Few polls have signaled Demo­crat­ic per­il this year quite like those in Col­or­ado, the purple state that twice helped push Barack Obama in­to the White House. Just look at Quin­nipi­ac’s sur­veys throughout 2013. They’ve found Obama’s pub­lic ap­prov­al in Col­or­ado ran­ging from the low 40s to the mid 30s; formerly high-fly­ing Gov. John Hick­en­loop­er in stat­ist­ic­al dead heats with some flawed GOP op­pon­ents; and Sen. Mark Ud­all well short of safe in his first reelec­tion bid. And, look­ing out to 2016, the statewide polls have Vice Pres­id­ent Joe Biden badly trail­ing in a match against one of the most po­lar­iz­ing Re­pub­lic­ans in Amer­ica, Sen. Ted Cruz of Texas.

Clearly Demo­crats have a prob­lem in this swing state. But a big­ger prob­lem for Demo­crats and Re­pub­lic­ans alike is polling it­self. It’s not that poll­sters are fail­ing to ac­cur­ately meas­ure what people really think. They are still quite good at that. The re­cur­ring and in­creas­ingly dis­rupt­ive prob­lem is that the polling in­dustry some­times struggles to reach the right mix of people.

This could be the case in Col­or­ado, where the sur­veys have a very dif­fer­ent makeup than what exit polls sug­gest is the typ­ic­al Col­or­ado elect­or­ate. In Quin­nipi­ac’s Col­or­ado sur­veys, whites without a col­lege de­gree out­num­ber col­lege-edu­cated whites as a share of the elect­or­ate — 43 per­cent to 35 per­cent in the most re­cent poll. But the exit polls that meas­ured who ac­tu­ally voted in re­cent state elec­tions have con­sist­ently found more de­gree-hold­ers than not among white voters. Those exits, which have var­ied, put the gap at 40 per­cent with de­grees to 37 per­cent without in 2012 and as high as 56 per­cent with and 25 per­cent without in 2010.

What this means is that the polls in Col­or­ado ap­pear to be put­ting too much weight on the views of Re­pub­lic­an-lean­ing voters, and thus ex­ag­ger­at­ing Demo­crats’ struggles in the state.

The dis­crep­ancy has not gone un­noticed by Col­or­ado Demo­crats. Poll “un­skew­ing” be­came a pop­u­lar — and widely mocked — par­tis­an hobby dur­ing the 2012 pres­id­en­tial elec­tion, but many of those cri­tiques fo­cused on the num­ber of self-iden­ti­fied Demo­crats or Re­pub­lic­ans, an at­ti­tude that changes over time, in­cluded in par­tic­u­lar polls. The num­ber of col­lege-de­gree hold­ers is, if not a fixed point, at least a demo­graph­ic meas­ure­ment that should be slower to evolve. Col­or­ado’s voters prob­ably won’t look much dif­fer­ent on that score in 2014 than they did col­lect­ively in 2012, 2010, or even 2008.

And al­though the dif­fer­ence is small, the makeup of those polled af­fects the res­ults. Quin­nipi­ac’s latest sur­vey showed Hick­en­loop­er mired around 45 per­cent sup­port for his reelec­tion, a danger zone for in­cum­bents. But the crosstabs — which break down the res­ults along demo­graph­ic and par­tis­an lines — show him draw­ing levels of sup­port from col­lege and non­col­lege whites close to where exit polls meas­ured him in the 2010 elec­tion. And he won that one with 51 per­cent of the state’s votes.

Now, take Col­or­ado’s polling is­sues and ap­ply them across the coun­try, and the sig­ni­fic­ance for 2014 and 2016 be­gins to settle in. How to meas­ure the views of the right mix of people is a grow­ing prob­lem for the polling in­dustry, es­pe­cially in an era of turnout-fo­cused elec­tions. Re­search has shown that all Amer­ic­ans are get­ting harder for poll­sters to reach but some more so than oth­ers, mak­ing it more dif­fi­cult to de­term­ine who is likely to vote and thus what an elec­tion’s fi­nal res­ult will look like. The GOP’s well-doc­u­mented polling mis­cues lead­ing up to the 2012 elec­tion stemmed partly from the be­lief that the elect­or­ate would be whiter than it proved to be.

Even the after-ac­tion polit­ic­al gold mines of exit polls are far from in­fal­lible. The rise of early vot­ing and vot­ing by mail has made it more dif­fi­cult to identi­fy voters’ “of­fi­cial” opin­ions. In Col­or­ado, more than three-quar­ters of voters cast their bal­lots by mail in 2012, and that might ex­plain some ma­jor vari­ance in the exit polls’ col­lege/non­col­lege splits. (Again, though, none has shown non­col­lege whites out­num­ber­ing col­lege whites.)

One way that some poll­sters try to com­bat these is­sues is by con­tact­ing their re­spond­ents us­ing lists of known voters, in­stead of di­al­ing phone num­bers ran­domly. Quin­nipi­ac is us­ing a dif­fer­ent but per­fectly reas­on­able meth­od to meas­ure Col­or­ado voters’ opin­ions. Its poll­sters ran­domly dial phone num­bers, gath­er in­form­a­tion such as race and edu­ca­tion­al at­tain­ment from re­spond­ents, and weight that sample of adults so that it re­flects what census data say Col­or­ado ac­tu­ally looks like. The adults who say they are re­gistered voters also an­swer polit­ic­al ques­tions, such as who they’ll sup­port for gov­ernor in 2014, and those re­gistered voters simply in­clude more blue-col­lar whites than col­lege-edu­cated ones, says Douglas Schwartz, dir­ect­or of the Quin­nipi­ac Uni­versity Poll.

“I don’t come at it from the vant­age point of, do we have too many non­col­lege-edu­cated whites,” Schwartz said. “I come from the vant­age point that we’re just weight­ing to what the census is telling us should be there.”

For sure, even if Quin­nipi­ac’s meas­ure­ments skew too blue-col­lar, the news is still not good for Col­or­ado Demo­crats. The in­sti­tute also meas­ured opin­ions about Hick­en­loop­er in 2012, be­fore he began strug­gling polit­ic­ally, and his ap­prov­al rat­ing has plunged by double di­gits since then.

“There’s lots of ob­ser­va­tions to be made, in­clud­ing that [the most re­cent Quin­nipi­ac] poll may not be per­fectly re­flect­ive of what 2014 may look like,” says Col­or­ado polit­ic­al ana­lyst Floyd Ciruli. “But it’s pick­ing up on what’s go­ing on out there,” as state Demo­crats have faced a back­lash this year cul­min­at­ing in two state Sen­ate re­calls and the de­feat of a Hick­en­loop­er-backed bal­lot meas­ure.

But in a polit­ic­al era when turnout is so im­port­ant, slight changes in poll samples can dial up or down the de­gree of that back­lash. Across the na­tion, and in Col­or­ado, poll­sters will struggle with how to cal­ib­rate this dial.

What We're Following See More »
Clinton Foundation Staffers Steered Biz to Bill
1 hours ago

"Two chief fundraisers for the Clinton Foundation pressed corporate donors to steer business opportunities to former President Bill Clinton as well, according to a hacked memo published Wednesday by WikiLeaks. The November 2011 memo from Douglas Band, at the time a top aide to Mr. Clinton, outlines extensive fundraising efforts that Mr. Band and a partner deployed on behalf of the Clinton Foundation and how that work sometimes translated into large speaking fees and other paid work for Mr. Clinton."

Chef Jose Andres Campaigns With Clinton
9 hours ago
White House Weighs in Against Non-Compete Contracts
10 hours ago

"The Obama administration on Tuesday called on U.S. states to ban agreements prohibiting many workers from moving to their employers’ rivals, saying it would lead to a more competitive labor market and faster wage growth. The administration said so-called non-compete agreements interfere with worker mobility and states should consider barring companies from requiring low-wage workers and other employees who are not privy to trade secrets or other special circumstances to sign them."

House Investigators Already Sharpening Their Spears for Clinton
11 hours ago

House Oversight Committee Chairman Jason Chaffetz plans to spend "years, come January, probing the record of a President Hillary Clinton." Chaffetz told the Washington Post: “It’s a target-rich environment. Even before we get to Day One, we’ve got two years’ worth of material already lined up. She has four years of history at the State Department, and it ain’t good.”

Clinton Super PAC Enters the House Fray
15 hours ago

Priorities USA, the super PAC aligned with the Clinton campaign, which has already gotten involved in two Senate races, is now expanding into House races. The group released a 30 second spot which serves to hit Donald Trump and Iowa Rep. Rod Blum, who is in a tough race to win re-election in Iowa's first congressional district. The super PAC's expansion into House and Senate races shows a high level of confidence in Clinton's standing against Trump.


Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.