NationalJournal.com
|
Search Sponsor:
|
MYSTERY POLLSTER
Don't Shoot The Pollsters
Reaction To NBC's Polling On The Public Option Misses The Whole Story
Like predictions? Here's an easy one: When the political winds shift and polls show bad news for a political candidate or cause, pollsters will become the object of partisan attack.
It happened just last week when a new NBC News poll found considerable public skepticism about Democratic proposals for health care reform. The office of House Speaker Nancy Pelosi put out a statement condemning the poll under the headline, "Bad Analysis, Worse Question," and the Huffington Post's Sam Stein noted the "longstanding ties to the health insurance industry" of Bill McInturff, the Republican pollster who conducts the NBC News/Wall Street Journal poll along with Democrat Peter Hart.
"McInturff is one of the most respected pollsters in Washington," Stein wrote, "and his work is often treated as straightforward and honest. But the pollsters' decision to alter the language of their August survey has raised the eyebrows of industry observers."
What provoked this criticism? Let's take a closer look.
In June, the NBC/Wall Street Journal poll devoted 14 questions to health reform, three to the proposed public option. The first question asked respondents to rate how "important" it is to "give people a choice of both a public plan administered by the federal government and a private plan for their health insurance." More than three-quarters considered this choice "extremely important" (41 percent) or "quite important" (35 percent).
They followed up with a question that presented two sentences arguing for the public option and two sentences arguing against, and asked respondents which they agreed with more. A narrower plurality (47 percent to 42 percent) expressed support for the public plan.
According to McInturff, he argued at the time that first question "doesn't measure support for the program, it simply measures Americans' desire to have more options," while the follow-up would better reflect the political "dialogue" that Americans would hear over the summer.
In their July poll, Hart and McInturff dropped the question about choice, opting instead to ask a simpler favor-or-oppose question about the public option that they intended to track, along with other more general measures, for the remainder of the debate. They found 46 percent of adults in favor and 44 percent opposed to "creating a public health care plan administered by the federal government that would compete directly with private health insurance companies."
They repeated the same question two weeks ago and found 43 percent in favor and 47 percent opposed, which NBC News described as a "shift" from the July results. The critical firestorm ensued, focusing on two complaints: First, that the observed change since July was not big enough to attain statistical significance and second, as Stein put it, that "the pollsters dropped the word 'choice' in their July and August polls."
The impetus for the criticism probably stems from the turn this story had taken after the release of the first poll in June. Liberal supporters of the public option embraced the results of the "importance" question as the best evidence of support for the proposal. "New Poll Shows 76% Support for Choice of Public Plan," read the headline of a report by Huffington's Stein. The pro-reform group known as the Progressive Change Campaign Committee prominently featured the statistic in a television advertisement and a Web site, WeWantThePublicOption.com. Many bloggers followed their lead.
Having invested so much in a single statistic, these groups' disappointment that the same poll now found a "plurality" opposed to the public option is understandable. Was it fair?
Critics were right to question the characterization of the difference between the July and August results as a "shift," since it was not large enough to attain statistical significance. Still, that conclusion was a very minor piece of NBC's reporting, and other surveys have tracked significant increases in general opposition to reform proposals over the summer months. Also, the most recent ABC News/Washington Post survey showed a 10 percentage point drop (from 62 percent to 52 percent) in support for "having the government create a new health insurance plan to compete with private health insurance plans" between June and August.
But the fury over "dropping the word 'choice'" has less merit. First, the pollsters did not so much drop one question as add another. They did not intend for their first question to serve as a measure of support for the public option. Their reporting never portrayed it as such, nor did it treat the new question as comparable to the old. Both ABC News polling director Gary Langer and blogger Nate Silver agree that the initial "importance" question was an inferior choice to measure support for the public option.
Yes, the newer question finds less support for the public option than similar favor-or-oppose questions asked by other pollsters. We could quibble over the wording (and NBC's Chuck Todd told Stein last week that the next NBC/Wall Street Journal poll will ask both the old and new questions), but as I wrote last week, there is no perfect way to ask about the public option, and I agree with Langer that the NBC question wording is "perfectly reasonable."
What I find unreasonable and unsupported are the attacks on McInturff's character, and the insinuation that he "biased the results" to serve a client, America's Health Insurance Plans, that opposes the public option. Among other things, this smear implies that Peter Hart -- who, as Stein recounts, serves numerous labor clients that support the public option -- was somehow asleep at the switch.
The NBC/Wall Street Journal poll works precisely because Hart and McInturff bring the experience of working on behalf of political campaigns and advocacy groups. My own work as a campaign pollster may be a source of bias, but a background in campaigns gives the pollster a different point of view. Campaign pollsters often have a better sense of the way public debates play out, the way opposing messages are framed and how public opinion can shift and change. That awareness adds special value to the NBC/Wall Street Journal poll and is a big reason why it continues to win respect from professionals in both parties.
Previously in Mystery Pollster
- Town Halls And Public Opinion (08/17/2009)
- Is Polling As We Know It Doomed? (08/10/2009)
- Huffington's Unintended Gift To Polling (08/03/2009)
- The Value Of Polling (07/27/2009)
- Pinning Down Worry-Free Health Care (07/20/2009)
Advertisement