It was, without doubt, the strangest polling story of the year.
In September, controversy swirled around public affairs and polling company Strategic Vision LLC following a rare censure from the American Association for Public Opinion Research and accusations of fraud from blogger Nate Silver. Yet the ensuing silence from Strategic Vision and its CEO, David Johnson, is even more troubling than the initial story.
I devoted a column to the subject three months ago, but here is a brief recap:
Strategic Vision LLC -- not to be confused with the highly respected research company Strategic Vision Inc. -- has released results from over 170 purported pre-election polls since 2004. Johnson told The Hotline three years ago that the company conducts surveys using live interviewers at two Florida call centers.
AAPOR requested information about two Strategic Vision polls in New Hampshire and Wisconsin along with those of 21 other organizations as part of a lengthy investigation of polling errors during the 2008 primaries. A year and a half later, after Strategic Vision failed to provide even cursory information about its response rates and weighting procedures, AAPOR issued a public statement in which it "raised objections" about Strategic Vision's failure to disclose "essential facts" as mandated by its Code of Professional Standards and Ethics.
(Disclosure: I'm an active AAPOR member and served on the AAPOR's Executive Council from 2006 to 2008.)
Shortly after AAPOR released its statement, Silver reacted by asking, rhetorically, whether Strategic Vision was "actually polling anyone at all." He proceeded to crunch some numbers and found a pattern in the trailing digits of the percentages reported in Strategic Vision polls indicating a "possibility of fraud." He later wrote two posts raising similar doubts about a Strategic Vision survey of students in Oklahoma, finally pronouncing himself "almost certain" that the company "is disreputable and fraudulent."
Two Sundays ago, the New York Times Magazine highlighted Silver's statistical takedown and included "forensic polling analysis" in its list of most noteworthy ideas of 2009. When Silver first posted his trailing digit analysis, I contacted some statistical wizards who privately expressed the same sentiments as those quoted by Wall Street Journal's Carl Bialik (here and here). The "forensic" results were troubling and merited further investigation, but were not conclusive.
Far more troubling are the ways in which the company has failed to document or, indeed, stand up for the validity of its numbers.
• After AAPOR released its statement, Strategic Vision abruptly stopped releasing polling results on its Web site. Compare that to the last quarters of 2005 and 2007, when the company released 16 and nine surveys, respectively.
• In late September, Johnson told reporters from the Wall Street Journal, The Hill, the Atlanta Journal-Constitution, St. Petersburg Times and the British publication Research that he planned legal action against AAPOR, its members, Silver, or all of the above. Yet both Silver and AAPOR President Peter Miller confirm to me that they have been notified of no such legal action.
• In late September, when Jim Galloway of the Atlanta Journal-Constitution asked to see standard cross-tabulations of the firm's data (something he had done at least twice previously without success), Johnson promised "to release all the crosstabs, and put an end to this right now.... That will squelch anybody from saying anything." Yet nearly three months later, no cross-tabs have been posted to their Web site or otherwise published.
"We intend to vindicate ourselves," Johnson told Politico's Ben Smith a few days after AAPOR announced its censure. Yet as 2009 draws to a close, what if anything has Johnson done to answer his critics or otherwise defend his company from Silver's accusations?
Just last week, Johnson penned an op-ed for the Journal-Constitution on Georgia politics. Johnson's biographical blurb at the bottom describes Strategic Vision as "a public relations and public affairs agency." In May, a similar op-ed described it as "a public affairs and polling company" (emphasis added). So perhaps Johnson has decided to get out of the polling business altogether (neither Johnson nor the editors of the Journal-Constitution responded to my requests for comment).
For some added perspective on this whole sorry episode, I turned to an individual who, unlike Nate Silver, has received absolutely no credit for his contribution. I'll call him "Deep Vision."
For more than three years, an unknown Democratic political consultant from Georgia has been contacting journalists at a half dozen or so publications (including me and Silver this year) pushing them to look into the possibility of something fishy at Strategic Vision. He had culled nearly every fact in the public domain about Strategic Vision. The information he shared initially struck me as suggestive of questionable methodology and aggressive salesmanship, but far short of evidence of fraud (though his research was especially helpful in a blog post I published in September).
Deep Vision recalls his frustration. "Most of these questions have been out there since 2004 but no matter how many times it was pitched to first local, and then national media, no one would write about it," he said. He also points out polls the firm conducted early in an election cycle, especially in Georgia, had real-world consequences. They helped shape elite opinion and candidate fundraising.
"The most troubling fact," he adds, "is that if Strategic Vision had simply responded to AAPOR, the press would still be covering their polls."
That's something for all of us that report on political polls to ponder as we head into the election year of 2010.