Ever since Election Day 2012, when Republicans lost the presidential popular vote for the fifth time in six cycles, in addition to two Senate seats and seven House seats, the party has been engaged in an extensive evaluation of its pollsters and the millions of dollars of survey research commissioned by its candidates and party committees each year. That effort — spearheaded by the Republican National Committee and the National Republican Congressional Committee, which are looking to reassert themselves as power players in D.C. — is a small but telling part of a tenuous process that seeks to transform the GOP into a party equipped to win among a diversifying electorate.
It’s an undertaking that, at times, has been as rocky as the party’s overall rebranding effort. NRCC officials and pollsters involved in the process this year portrayed the meetings as largely positive, but they have been marked by more debate than has been previously reported, and by persistent disagreement among some that the party’s polls last year were all that far off.
“There’s a little bit of “˜Oh, we’re still going to do it this way,’ “ said Jon McHenry of North Star Opinion Research in Alexandria, Va., while noting that such sentiment was not the consensus of the NRCC get-togethers. “We all had a stake in making sure the NRCC did things in the best way possible.”
Some pollsters, though, are reticent to give up too much autonomy to the national party or to spill trade secrets to rivals. While pollsters work together in some cases — polling for the campaign committees’ independent-expenditure units is one notable example — they often compete against one another when it comes to candidate business. “We are totally on board,” said Kellyanne Conway, president and CEO of the polling company, inc./WomanTrend, who attended a gathering at the RNC. “But as I sat in that meeting, I thought, “˜Wait a second, if we do all this, you’re actually cutting into my competitive advantage.’ “
Moreover, the underlying realities that are challenging pollsters — for example, that technology is making some voters harder to reach by landline telephone — aren’t that different from the GOP’s struggles to reach certain segments of the electorate. In fact, some of the party’s polling problems in 2012 were rooted in erroneous assumptions about the composition of the electorate. Most Republican pollsters who spoke with National Journal throughout this year said they thought the electorate would be more favorable to their party — meaning whiter, older, and overall more Republican — than the voters who showed up to cast ballots on Election Day. It was a vicious cycle for the party: Incorrect assumptions resulted in falsely optimistic poll numbers during the campaign, and those numbers served to reinforce the GOP’s projections about turnout.
But turnout rates in 2012 actually fell among white voters. Some on the right have suggested that goosing turnout among the party’s base would be a more effective strategy than trying to appeal to minorities or younger voters. The GOP establishment has taken the opposite view: The RNC’s Growth and Opportunity Report, which party officials hoped would serve as a blueprint for a political comeback, specifically endorsed comprehensive immigration reform as a starting point for appealing to Hispanics and recommended efforts to reach out to young people.
In many ways, the polling reboot is emblematic of this broader debate within the party: Does the GOP need to adapt to changing demographics to win national elections, and, if so, to what extent?
The NRCC and the RNC have both produced lists of recommendations. Most GOP pollsters who spoke with National Journal endorsed the proposals as a set of best practices, although they differed on the size and scope of the party’s polling problems. Some said the recommendations are incomplete, emphasizing some measures over others for fixing the party’s polls. Other GOP pollsters who weren’t involved in the NRCC process see the effort as too insular. And most of the recommendations would result in higher costs that would have to be passed along to clients.
The overriding theme of these conversations, however, is of a party trying to play catch-up with its Democratic rival when it comes to survey research and data. That this deficit has come amid broader, longer doldrums for the GOP is hardly coincidental.
“Democrats are at the graduate level,” one Republican pollster groaned this summer. “We’re still in kindergarten when it comes to the behavioral side of voter opinion.”
PRACTICES MAKE PERFECT?
When Rep. Greg Walden, R-Ore., the chairman of the NRCC, stood before the House GOP Conference on May 15 to present recommendations for the party’s polling in the 2014 cycle, his report represented the culmination of an unprecedented months-long reassessment spearheaded by the committee.
A more detailed memorandum had gone out to most of the party’s major polling firms two days before Walden’s presentation, GOP sources said. Both the memo and Walden’s presentation contained a series of best practices for GOP survey researchers developed after months of meetings and input with pollsters used by the NRCC and Republican House candidates.
Some Republicans have faulted the party’s polling — more specifically, the modeling that built assumptions around what the 2012 electorate would look like — for their poor performance last year. Had the polling more accurately reflected the electorate, it could have enabled campaigns, the campaign committees, and other outside groups to make better-informed decisions to maximize the value of their investments, they argue.
But others deny the problem is that serious. They contend the party’s polling (or their polling, at least) was accurate — until some races slipped away at the end. While internal polls two weeks before the election may not jibe with election results, they were accurate at the time, these pollsters contend. And although polling immediately before the election would produce more-accurate numbers, it would have little value other than to create expectations; the sorts of decisions that polls inform, such as television ads and other investments, are already made before the final weekend of the campaign.
The Republicans’ congressional committee led a comprehensive review aimed at addressing many of the challenges facing pollsters on both sides of the aisle. Americans are becoming harder to reach by telephone, particularly landlines. And as Republicans continue to struggle to attract nonwhite voters, who tend to be even more difficult to reach than whites, including the right number of minority voters and in the right proportion is essential to producing an accurate poll.
To accomplish that, the two entities driving the GOP re-branding have each made a number of recommendations. The NRCC asked Republican polling firms to use larger sample sizes, call more cell phones, and conduct bilingual interviews in districts with significant Hispanic populations. The RNC autopsy also asked the party’s survey researchers to experiment with registration-based sampling methods — calling phone numbers randomly selected from a known list of registered voters — that are already used by some, but not all, campaign pollsters. “The thing is, with a listed sample, you know so much more about these people,” said Alex Lundry, a pollster with TargetPoint Consulting in Alexandria, Va., and the head of Mitt Romney’s 2012 data team. “That helps enrich the analysis of the completes you have; it helps you do a better analysis of nonresponse bias.”
The conventional argument for using registration-based sampling centers on pollsters’ attempts to identify which voters are likely to turn out on Election Day. Some pollsters will start with the sample that includes only voters whom public records show cast ballots in recent elections, provided they were eligible to do so. That, the voter-list pollsters argue, can reduce the number of nonvoters in their samples.
Lundry’s argument for registration-based sampling takes that further. Completing an interview off a voter file gives you demographic information — and, potentially, consumer data that can be matched to it. Furthermore, because you have more information about the people with whom you’re speaking, you also know more about the people whom you aren’t reaching as effectively by telephone.
But some firms are holding onto random-digit dialing for sampling, most notably Public Opinion Strategies, the firm that polled for Romney last year. At an event in Washington earlier this month, Bill McInturff, a partner at POS, defended random-digit dialing and said that using voter files for sampling could lead to the same underestimation of certain blocs of voters that plagued the party’s 2012 polls.
“When you look at the voter file of [people] who actually voted, what you find is, it doesn’t match the exit polls,” McInturff said.
“It’s much older, whiter, less ethnic; and my fight inside the party’s been, “˜Yeah, because those are the folks they can match [to voter files]!’ That means there are thousands of voters that you can’t match that you’re missing. Guess what: They’re poorer, younger, and ethnic and nonwhite.”¦ There’s a lot of other folks who can vote that don’t get matched on a voter file because we can’t pick ‘em out because we don’t have a phone number.”
While the fight over sampling methods rages — the RNC report recommended side-by-side studies of both methods — the NRCC’s suggestion of larger sample sizes may help solve the problem. It would allow campaigns and other groups to have a better, more significant understanding of harder-to-reach demographic groups, such as young people, minorities, and independents. Moreover, the segments of these cohorts reached may significantly differ from the population of the cohort at large. In other words, younger voters who participate in phone polls (particularly landline phone polls) differ from the overall younger electorate — it’s thought they are more likely to be living with their parents — to a greater degree than seniors who participate in polls differ from the overall senior electorate. Because young people, minorities, and independents tend to be underrepresented to begin with, the practice of weighting the value of their responses to meet a set threshold may actually increase the error in the poll.
The party has recommended increasing the number of cell-phone interviews because, for more than a third of Election Day 2012 voters, their mobile device was their only telephone. This was also true in key battleground states. Some 34 percent of Ohio voters, 32 percent of Virginia voters, 35 percent of North Carolina voters, and 34 percent of Florida voters were cell-only. Cell-only Americans also tend to be younger and disproportionately nonwhite. Underrepresenting them in Republican polls is likely to overstate the GOP candidate’s ballot position.
GOP pollster Glen Bolger wrote in a November 2012 memorandum that the party must start surveying more voters using cell phones. “We have been doing cell-phone interviewing for several cycles now, but really increased it this year with cell-phone interviewing not only at the national and statewide level, but also at the congressional level. What we do know now is that 20 percent of the interviews with cell phones was not enough,” Bolger wrote.
Meanwhile, conducting more interviews in Spanish is designed to get a more accurate depiction of Hispanic voters and their percentage as a share of the electorate, according to Bolger. It isn’t enough for Republicans to just increase the number of Hispanics in their surveys. Boosting the percentage of Latinos who complete a phone survey conducted in English may only exacerbate a poll’s deficiency: Those who would be more comfortable answering questions in English may behave differently at the ballot than Hispanics who don’t want to participate in an English-only survey.
Each of these recommendations represents a response to changes in demographic trends and in the way people communicate with one another. But each recommendation also represents a significant increase in survey costs that would be passed on to clients.
The NRCC, in advocating for these more expensive protocols, says it is willing to pay higher prices for more-reliable polling. “I think there is an acknowledgment based on all of these things that cost is going to go up a little bit, but I think there’s also an acknowledgment from us on the consumer side of this and from our candidates and campaigns that the cost part of this is fine so long as the data and the stuff you’re getting back is accurate,” said NRCC Political Director Rob Simms.Still, not every Republican polling firm is singing from the same hymnal. Some pollsters said their results were accurate, and that the reevaluation process is based on the incorrect assumption that, because the party’s standard-bearer lost when his pollster felt confident about his chances, the party’s polls on the whole were faulty.
One GOP pollster described a conference call with Neil Newhouse, Bolger’s partner at Public Opinion Strategies and the Romney campaign’s lead pollster, the Friday before the election in which Newhouse said that each of the 28 latest public polls showed Romney leading among independents. The reason the entire GOP polling industry was asked to come together, the pollster said, was because the Romney campaign thought it had a plausible path to victory based on faulty data. “I don’t think if Mitt Romney had won based on POS’s polling, they’d be in that room,” the pollster said.
Republicans hope their efforts to improve the party’s polling will help close the overall data gap. “What’s important to remember is that campaigns are already increasing the amount of data that they look at,” said Brock McCleary, the NRCC’s polling director during the 2012 election cycle and the founder of a new firm that focuses on automated phone polling. “Polling should almost be seen in the context of an entire operation of data.”
Obama’s 2012 campaign provides an interesting marker for the GOP’s efforts. Republicans clearly would like to replicate the Democrats’ successes, but it’s less clear that the NRCC- and RNC-led polling recommendations will close that gap.
Lundry’s position with the Romney camp gave him a unique, inside perspective on campaign data in 2012. “I strongly believe that we had access to about 90 to 95 percent of the data technology and tools the Obama campaign had,” Lundry said, referencing postelection reports on the opposition’s operations. “The extraordinary difference was a lack of integration.”
Lundry thinks the polling reboot comes up short in erasing that disparity entirely, but he cited reports that the NRCC would begin using polls to better inform its television advertising spending. “I think that’s extraordinarily important,” he said. “TV’s the biggest line item in the budget but the least data-driven.”
Lundry’s latest project, the data-tracking firm Deep Root Analytics, is set to provide Republican candidates and outside groups with even more-detailed information — culled anonymously from viewers’ cable set-top boxes — to inform ad buying and targeting of specific demographics. In a separate interview in August, Lundry said he hopes to integrate that information with polling at the designated market-area level. The New York Times reported last month that Deep Root Analytics was working for New Jersey Gov. Chris Christie’s reelection campaign. “We’re trying to have as comprehensive a look at the paid-media ecosystem and trying to understand what’s working and what’s not,” Lundry said.
The NRCC also acknowledged a lack of integration between its data and decision-making.
“It’s not that Democrats have the monopoly on really good data, and we’re floundering in the wilderness, if you will,” said the NRCC’s Simms. “I think where the Democrats have excelled, particularly in  and even , is creating the infrastructure to truly utilize the data and have the data drive their decision-making process for everything the campaign does. And that’s our goal, that’s our motivation. So the stuff we’re doing with polling, for example, is starting with the data side, where we’re going to model our districts in-house, which is going to affect how we poll them, which is going to affect how we micro-target them, which is going to affect how we run the campaigns. So, for us, the ultimate objective is for all of our candidates to run the best, most data-driven campaign they can run. And I think the Democrats really excelled at that last cycle and embraced it. And I’m not sure that we did.”
Setting up the necessary infrastructure is easier if you’re a president with four years to plot your reelection campaign. Since the eventual 2016 GOP presidential nominee will have less than a year to build a campaign apparatus, the role as change agent has fallen to the RNC and NRCC. And — so far at least — their involvement has some Republicans cautiously optimistic about their party’s chances to turn things around, despite the differences among its consultant class. “Only crisis compels change,” McCleary said.