Pros and Cons
John Sides, a political-science professor at George Washington University and the coauthor of the blog The Monkey Cage, is one of the more prominent proponents of using polling averages--and a critic of press coverage that doesn't. "You're better off looking at averages, because any individual poll may be different from the truth because of sampling error and any idiosyncratic decisions that pollsters make," he said.
Sides believes that news coverage of campaigns tends to overemphasize some polls at the expense of others. In some cases, a poll is considered newsier because it shows something different than the balance of other polling in the race. In other words, polls that are outliers are given more attention than polls that hew more closely to the average, and those outlier polls are more likely to be inaccurate, Sides argues.
"I'm not overly optimistic that the averages are going to become a more important factor in news coverage. I think there are still strong incentives to seek drama where you can find it. And that may mean chasing an outlier," Sides said. "I would like to think that it would start to creep in at the margins. So instead of saying, 'Some polls say ___,' it may say, 'A new poll showed ___, but other polls haven't showed that yet.' "
But, as this year's results show, the averages aren't perfect, and they all showed a closer race than the actual outcome. Comparing polls that accurately predicted the election--the final poll from the Pew Research Center, for example--to the poll averages at the time would have made those more accurate polls appear to be outliers.
Part of that problem, at least when it comes to the national presidential race, were the daily tracking polls from Gallup and automated pollster Rasmussen Reports. Both firms reported results that were biased in favor of Romney this cycle, but by publishing a new result every day, their polls could be overrepresented in the averages. "The one sort of Achilles' heel of the regression trend line that we've done classically on our charts, there are two pollsters that contribute most of the data points," said Pollster's Blumenthal. "Not only does that make the overall aggregate off, it can also create apparent turns in the trend line that are [because] we've had nothing but Gallup and Rasmussen polls for the last 10 days."
In addition to the ubiquitousness of surveys from some firms, poll aggregators also worry that partisans may try to game the system by releasing polls with greater frequency, or by skewing or fabricating results. A prominent Democratic strategist told National Journal last month that some outside groups conducted polls in presidential swing states and released them to the public as a means of countering polls from Rasmussen Reports that were less favorable to President Obama. Blumenthal told National Journal that his biggest fear was not the increase in partisan polls but the possibility that groups would release rigged or fabricated results, as outfits like Strategic Vision and Research 2000 apparently have over the past handful of years, to influence the averages. "My biggest concern over the last two or three years has been the potential for the repeat of something like that," Blumenthal said.
"Champagne" of Polls
The blog post written by Gallup's Newport after the election demonstrates another source of opposition to poll averages: the pollsters themselves. Pollsters all make choices about how best to sample the probable or likely electorate, and those choices vary. Additionally, more expensive live-caller polls compete in the averages with cheaper automated-phone and Internet polls that may not make the same efforts to obtain random samples of voters; merits aside, those live-caller pollsters surely want to protect their businesses from less expensive competitors. Moreover, news organizations that spend tens of thousands of dollars to conduct a poll are likely to report and trumpet their poll's results over other surveys.
"If you're merely an information aggregator, it's very hard for me to see how you're adding value to the proposition," Gary Langer, whose firm Langer Research Associates produces polls for ABC News, told National Journal in a phone interview last month. "Averaging polls is like averaging champagne, Coca-Cola, and turpentine," Langer added.
Overall, 2012 brought more attention than ever to poll aggregators, with their methods becoming more sophisticated. But where do they go from here?
"I don't think there are great advances in averaging or modeling horse-race polling data," Blumenthal said. "We are ultimately reliant on the quality of the data that's collected."
There is evidence that data are becoming less reliable, but supporters of using polling averages argue the underlying changes that are leading to more variable polls bolster their case for using averages instead of individual poll results. "It's a powerful piece of information, and it's a very good piece of information, and I think it's better than any one single poll in terms of using it as a data point to analyze a race," said RCP's McIntyre.
The questions remain open over how much attention the averages will get in the next election--and how much attention they deserve. But, for the horse-race media, 2016 is right around the corner, and some aggregators have already started keeping score.
Get the latest news and analysis delivered to your inbox. Sign up for National Journal's morning alert, Wake-Up Call, and afternoon newsletter, The Edge. Subscribe here.
Leave A Comment