Skip Navigation

Close and don't show again.

Your browser is out of date.

You may not get the full experience here on National Journal.

Please upgrade your browser to any of the following supported browsers:

Are Issue Polls 'Next To Useless'? Are Issue Polls 'Next To Useless'?

This ad will end in seconds
 
Close X

Not a member? Learn More »

Forget Your Password?

Don't have an account? Register »

Reveal Navigation
 

 

Legacy Content / MYSTERY POLLSTER

Are Issue Polls 'Next To Useless'?

They Mean More The More Of Them You Consult

November 23, 2009

Last week, Time columnist Joe Klein had seen one issue poll too many.

"Polling on issues is next to useless," he blogged, "especially on issues as emotionally complicated as wars and as technically complicated as health care reform." Surprisingly, many pollsters, including yours truly, see some merit in Klein's argument about the way the news media typically report poll questions on public policy issues, even if we might not be ready to cast away such polls altogether.

Let's start with where we agree. Klein was talking about two new survey questions, one from CNN and the Opinion Research Corp. showing plurality opposition (49 percent to 46 percent) to the health care bill just passed by the House, and another from ABC News and the Washington Post showing a majority (52 percent) concluding that the war in Afghanistan has not been worth fighting.

 

Klein then speculated, reasonably, that different questions asked about the same subjects would likely tell a different story. "The only safe conclusion from these particular polls," Klein concludes, is that "the public has mixed feelings on Afghanistan and health care reform. Brilliant! I have mixed feelings, too."

His objection then, is less about specific survey results than the screaming headlines they typically produce ("Public Opposes Health Bill. Public Opposes War") and the false sense of certainty they convey. Fair enough. I have offered similar caution about the limitations of issue questions.

When I posted Klein's comments at Pollster.com, I heard from a handful of campaign pollsters via Twitter who quarreled with his blanket dismissal of issue polling but generally agreed with his critique of the way the news media often reports on survey results.

Democratic pollster Jason Boxt, for example, finds it "infuriating when polls are misrepresented (this am on Morning Joe, for example)." Republican micro-targeter Alex Lundry chimed in to say that any agreement with Klein "would be limited to media polls."

"Klein reminds us why most campaign pollsters don't ask [questions] like media pollsters do," writes Republican pollster Matt Dabrowski. "Speculative policy is not our deal. Rather, it is how policy might move elections."

Having spent much of my career conducting internal polling for political campaigns, I tend to agree. Klein is also right that poll questions are flawed measures of attitudes on public policy issues.

But we all still see value in looking at polling data (even Klein ends his blog post confessing that he, too, is addicted). The danger is labeling any one result to any one question as "The Truth." To get closer to the reality of public opinion on an issue like health care, we need look at a lot of different questions asked by many different pollsters.

Consider, for example, the CNN poll that stoked Klein's fire. It included quite a few health care questions -- roughly 20 in all (released in two batches by CNN) -- "asked in a different way" about health care reform. These various measures, while undoubtedly imperfect, collectively provide a deeper picture of the rich, sometimes conflicting attitudes and values that make up "public opinion" on health care reform.

Now consider one specific result from that poll in the context of the "Public Opposes Health Bill" headline that Klein imagines and that many recent poll results support. Immediately after their favor-or-oppose question about the recently passed health reform bill, CNN's pollsters asked a new follow-up question that others have not. They found that while a third of all adults (34 percent) say they oppose the bill because "its approach toward health care is too liberal," 10 percent oppose it because it is "not liberal enough."

As Texas Tech University professor Alan Reifman points out, those results mean 56 percent of Americans "favor either the House-passed version of health care reform or something further to the left."

A check of the Nexis news search shows that CNN broadcast results of the follow-up question as part of their poll stories several times on the day they were released. And the story that ran on CNN's online Political Ticker featured the same numbers prominently, including this characterization from CNN Polling Director Keating Holland: "That may indicate that a majority opposes the details in the bill, but also that a majority may approve of the overall approach taken by House Democrats and President Obama."

Ironically, the story featured a different headline than what Klein had imagined: "Americans split on health care reform."

More Legacy Content
Job Board
Search Jobs
Professional Development Program Engineer
American Society of Civil Engineers | Farmington Hills, MI
Biomedical Service Internship Position
American Society of Civil Engineers | Flint, MI
Deputy Director of Transit Operations
American Society of Civil Engineers | San Jose, CA
Structural Engineer
American Society of Civil Engineers | New Haven, CT
Transportation Planner
American Society of Civil Engineers | Salinas, CA
Assessment and Remediation Team Lead
American Society of Civil Engineers | Regina, SK
Quality Systems Manager
American Society of Civil Engineers | Greensboro, NC
Quality Engineer
American Society of Civil Engineers | Attica, IN
Senior Water Resource Project Manager
American Society of Civil Engineers | Fairfax, VA
Civil Engineering
American Society of Civil Engineers | Steamboat Springs, CO
Entry Level Geotechnical Engineer
American Society of Civil Engineers | Albany, NY
Sales Engineer-Process Automation/Instrumentation
American Society of Civil Engineers
Senior Project Manager- Transportation
American Society of Civil Engineers | San Antonio, TX
Materials Engineer 2
American Society of Civil Engineers | IL
Land Surveyor
American Society of Civil Engineers
 
Comments
comments powered by Disqus