Political Pollsters Still Don’t Know What They’re Doing

Many of the problems behind inaccurate surveys in last year’s presidential election are surfacing again in the midterm races.

TO GO WITH AFP STORY BY JULY FRAYSSE - A woman makess a phone call with an iPhone in Bordeaux on May 3, 2013. 
AFP/Getty Images
Scott Bland
Add to Briefcase
See more stories about...
Scott Bland
Dec. 8, 2013, midnight

Few polls have signaled Demo­crat­ic per­il this year quite like those in Col­or­ado, the purple state that twice helped push Barack Obama in­to the White House. Just look at Quin­nipi­ac’s sur­veys throughout 2013. They’ve found Obama’s pub­lic ap­prov­al in Col­or­ado ran­ging from the low 40s to the mid 30s; formerly high-fly­ing Gov. John Hick­en­loop­er in stat­ist­ic­al dead heats with some flawed GOP op­pon­ents; and Sen. Mark Ud­all well short of safe in his first reelec­tion bid. And, look­ing out to 2016, the statewide polls have Vice Pres­id­ent Joe Biden badly trail­ing in a match against one of the most po­lar­iz­ing Re­pub­lic­ans in Amer­ica, Sen. Ted Cruz of Texas.

Clearly Demo­crats have a prob­lem in this swing state. But a big­ger prob­lem for Demo­crats and Re­pub­lic­ans alike is polling it­self. It’s not that poll­sters are fail­ing to ac­cur­ately meas­ure what people really think. They are still quite good at that. The re­cur­ring and in­creas­ingly dis­rupt­ive prob­lem is that the polling in­dustry some­times struggles to reach the right mix of people.

This could be the case in Col­or­ado, where the sur­veys have a very dif­fer­ent makeup than what exit polls sug­gest is the typ­ic­al Col­or­ado elect­or­ate. In Quin­nipi­ac’s Col­or­ado sur­veys, whites without a col­lege de­gree out­num­ber col­lege-edu­cated whites as a share of the elect­or­ate — 43 per­cent to 35 per­cent in the most re­cent poll. But the exit polls that meas­ured who ac­tu­ally voted in re­cent state elec­tions have con­sist­ently found more de­gree-hold­ers than not among white voters. Those exits, which have var­ied, put the gap at 40 per­cent with de­grees to 37 per­cent without in 2012 and as high as 56 per­cent with and 25 per­cent without in 2010.

What this means is that the polls in Col­or­ado ap­pear to be put­ting too much weight on the views of Re­pub­lic­an-lean­ing voters, and thus ex­ag­ger­at­ing Demo­crats’ struggles in the state.

The dis­crep­ancy has not gone un­noticed by Col­or­ado Demo­crats. Poll “un­skew­ing” be­came a pop­u­lar — and widely mocked — par­tis­an hobby dur­ing the 2012 pres­id­en­tial elec­tion, but many of those cri­tiques fo­cused on the num­ber of self-iden­ti­fied Demo­crats or Re­pub­lic­ans, an at­ti­tude that changes over time, in­cluded in par­tic­u­lar polls. The num­ber of col­lege-de­gree hold­ers is, if not a fixed point, at least a demo­graph­ic meas­ure­ment that should be slower to evolve. Col­or­ado’s voters prob­ably won’t look much dif­fer­ent on that score in 2014 than they did col­lect­ively in 2012, 2010, or even 2008.

And al­though the dif­fer­ence is small, the makeup of those polled af­fects the res­ults. Quin­nipi­ac’s latest sur­vey showed Hick­en­loop­er mired around 45 per­cent sup­port for his reelec­tion, a danger zone for in­cum­bents. But the crosstabs — which break down the res­ults along demo­graph­ic and par­tis­an lines — show him draw­ing levels of sup­port from col­lege and non­col­lege whites close to where exit polls meas­ured him in the 2010 elec­tion. And he won that one with 51 per­cent of the state’s votes.

Now, take Col­or­ado’s polling is­sues and ap­ply them across the coun­try, and the sig­ni­fic­ance for 2014 and 2016 be­gins to settle in. How to meas­ure the views of the right mix of people is a grow­ing prob­lem for the polling in­dustry, es­pe­cially in an era of turnout-fo­cused elec­tions. Re­search has shown that all Amer­ic­ans are get­ting harder for poll­sters to reach but some more so than oth­ers, mak­ing it more dif­fi­cult to de­term­ine who is likely to vote and thus what an elec­tion’s fi­nal res­ult will look like. The GOP’s well-doc­u­mented polling mis­cues lead­ing up to the 2012 elec­tion stemmed partly from the be­lief that the elect­or­ate would be whiter than it proved to be.

Even the after-ac­tion polit­ic­al gold mines of exit polls are far from in­fal­lible. The rise of early vot­ing and vot­ing by mail has made it more dif­fi­cult to identi­fy voters’ “of­fi­cial” opin­ions. In Col­or­ado, more than three-quar­ters of voters cast their bal­lots by mail in 2012, and that might ex­plain some ma­jor vari­ance in the exit polls’ col­lege/non­col­lege splits. (Again, though, none has shown non­col­lege whites out­num­ber­ing col­lege whites.)

One way that some poll­sters try to com­bat these is­sues is by con­tact­ing their re­spond­ents us­ing lists of known voters, in­stead of di­al­ing phone num­bers ran­domly. Quin­nipi­ac is us­ing a dif­fer­ent but per­fectly reas­on­able meth­od to meas­ure Col­or­ado voters’ opin­ions. Its poll­sters ran­domly dial phone num­bers, gath­er in­form­a­tion such as race and edu­ca­tion­al at­tain­ment from re­spond­ents, and weight that sample of adults so that it re­flects what census data say Col­or­ado ac­tu­ally looks like. The adults who say they are re­gistered voters also an­swer polit­ic­al ques­tions, such as who they’ll sup­port for gov­ernor in 2014, and those re­gistered voters simply in­clude more blue-col­lar whites than col­lege-edu­cated ones, says Douglas Schwartz, dir­ect­or of the Quin­nipi­ac Uni­versity Poll.

“I don’t come at it from the vant­age point of, do we have too many non­col­lege-edu­cated whites,” Schwartz said. “I come from the vant­age point that we’re just weight­ing to what the census is telling us should be there.”

For sure, even if Quin­nipi­ac’s meas­ure­ments skew too blue-col­lar, the news is still not good for Col­or­ado Demo­crats. The in­sti­tute also meas­ured opin­ions about Hick­en­loop­er in 2012, be­fore he began strug­gling polit­ic­ally, and his ap­prov­al rat­ing has plunged by double di­gits since then.

“There’s lots of ob­ser­va­tions to be made, in­clud­ing that [the most re­cent Quin­nipi­ac] poll may not be per­fectly re­flect­ive of what 2014 may look like,” says Col­or­ado polit­ic­al ana­lyst Floyd Ciruli. “But it’s pick­ing up on what’s go­ing on out there,” as state Demo­crats have faced a back­lash this year cul­min­at­ing in two state Sen­ate re­calls and the de­feat of a Hick­en­loop­er-backed bal­lot meas­ure.

But in a polit­ic­al era when turnout is so im­port­ant, slight changes in poll samples can dial up or down the de­gree of that back­lash. Across the na­tion, and in Col­or­ado, poll­sters will struggle with how to cal­ib­rate this dial.

What We're Following See More »
FIRST TIME SINCE COLD WAR
Nuclear Bombers Preparing to Go Back on 24-Hour Alert
9 hours ago
THE LATEST

"The U.S. Air Force is preparing to put nuclear-armed bombers back on 24-hour ready alert, a status not seen since the Cold War ended in 1991...Putting the B-52s back on alert is just one of many decisions facing the Air Force as the U.S. military responds to a changing geopolitical environment that includes North Korea’s rapidly advancing nuclear arsenal, President Trump’s confrontational approach to Pyongyang, and Russia’s increasingly potent and active armed forces."

Source:
TRUMP’S ATTORNEY WAS SET TO TESTIFY ON WEDNESDAY
Senate Intel Postpones Testimony by Cohen
2 days ago
THE LATEST
AMENDMENT WOULD HAVE PREVENTED CONSIDERATION
Senate Rejects Effort to Nix SALT Tax Changes
3 days ago
THE LATEST

"Senate Democrats on Thursday failed in their first attempt to save the state and local tax deduction, which helps many residents of California and other high-cost states reduce their federal income tax bills. The Republican-controlled Senate voted 52-47 to reject an amendment that would have prevented the Senate from considering any bill that repeals or limits the deduction as part of a planned tax overhaul."

Source:
INTERVIEWED BY COMMITTEE STAFF
Lewandowski Meets with Senate Intelligence Committee
3 days ago
THE LATEST

"President Donald Trump's former campaign manager Corey Lewandowski appeared on Capitol Hill for a closed-door interview with the Senate intelligence committee Wednesday, according to a source familiar with the matter. Lewandowski is the latest senior official in Trump's orbit who has met with the committee as part of its investigation into Russian election meddling and possible collusion with the Trump campaign."

Source:
FISHING EXPEDITION
Some Members Seek to Wrap Up Russia Investigations by Year’s End
4 days ago
THE LATEST

"A growing number of key Republicans are sending this message to the leaders of the congressional committees investigating potential Trump campaign collusion with the Russians: Wrap it up soon. In the House and Senate, several Republicans who sit on key committees are starting to grumble that the investigations have spanned the better part of the past nine months, contending that the Democratic push to extend the investigation well into next year could amount to a fishing expedition."

Source:
×
×

Welcome to National Journal!

You are currently accessing National Journal from IP access. Please login to access this feature. If you have any questions, please contact your Dedicated Advisor.

Login