Skip Navigation

Close and don't show again.

Your browser is out of date.

You may not get the full experience here on National Journal.

Please upgrade your browser to any of the following supported browsers:

Reveal Navigation

Polls And The Google Zeitgeist Polls And The Google Zeitgeist Polls And The Google Zeitgeist Polls And The Google Zeit...

share
This ad will end in seconds
 
Close X

Not a member? Learn More »

Forget Your Password?

Don't have an account? Register »

Reveal Navigation
 

 

Legacy Content / MYSTERY POLLSTER

Polls And The Google Zeitgeist

One Look At How Media Consumers Use Polling

December 7, 2009

Last week, Google announced the 2009 update of its much-discussed Zeitgeist index: lists of the fastest-rising and fastest-falling terms entered into the Google search engine in the United States. As Politico's Ben Smith noted, four of the 10 fastest-falling terms this year were names from Campaign 2008. Suspecting that "polls" might also be high on the list, I checked with the folks at Google; while they would not share a specific number, they confirmed that "polls" falls somewhere "in the top 25."

Comment on this column at Pollster.com.

Since we take data seriously here, I should note that these sorts of lists have been criticized as scrubbed, fuzzy and often misunderstood. Google cleans up the lists to "filter out spam and repeat queries" (as well as adult keywords) to create lists that "best reflect 'the spirit of the times.'" As the Wall Street Journal's Carl Bialik noted two years ago, since Google reports only the fastest-rising and -falling terms, its Zeitgeist lists are often misinterpreted as "as accurate portrayals of Americans' favorite queries."

 

For my purposes, however, Google's fastest-falling list is helpful, since it confirms a pattern I know all too well: the massive spike in interest in poll results during the final two months before general elections, especially elections for president.

I have followed the data in the chart below for several years using the Google Trends tool. The top part of the chart shows that just before the last two presidential elections, searches for the term "polls" were 15 to 20 times more frequent than the overall average of such searches since 2004. You can also see a much smaller surge in interest just before the 2006 general elections and higher-than-average interest during the heated presidential primary contests of early 2008.

The bottom of the chart plots something different: the number of times the word "polls" appeared in stories that Google indexes for its news search feature. So the frequency of what mainstream media sources pushed out -- references to the word "polls" in news stories -- closely matches what their readers pulled in with Internet searches. Both measures surge near elections.

Whatever the source, this pattern implies something important about how ordinary news consumers think about "polls." Some of my colleagues note that political horse-race polling is only a small part of the work of survey research, and they argue that measuring candidate preference is the least important purpose of pre-election polling. Some urge their readers to "throttle back on the horse race." To the consumers, however, horse-race measurements define political polling and motivate much of their interest in it.

That pattern becomes even clearer when we examine searches for some of the popular polling brands. Using the Google Trends tool, I checked on a number of the best-known names in the polling business. Not surprisingly, the winner in my queries is Gallup. Few polling brands come close in terms of Google searches. The runner-up in my informal data dive may be something of a surprise: Rasmussen Reports. According to Google Trends, since 2004 Google users searched on "Rasmussen" and "poll" slightly more than half as often as they searched on "Gallup" and "poll."

The chart above, however, shows something of a divergence between the pattern of user searches and news media references. In the mostly mainstream media outlets that Google indexes for its news search feature, mentions of Gallup polls are far more constant -- nowhere near the same surge and decline around election time -- and appear more numerous, than references to Rasmussen polling.

That pattern comports with experience: Gallup is, for good reason, a source that journalists and editors know and trust. Leading media outlets, including ABC News, the Washington Post, the New York Times and AP continue to forbid publication of the results of automated telephone surveys like those produced by Rasmussen. Consistent differences in Rasmussen's results on key measures like presidential approval continue to raise suspicions among journalists and partisans.

Take these Google search data queries with a grain of salt. You can produce different results by changing the wording. Also, as Carl Bialik implies, Google's disclosure of the sampling, modeling and estimation procedures used to produce these data are far from crystal-clear.

But with those caveats in mind, note that intriguing trend in the second chart above: User searches on "Rasmussen" and "poll" moved ahead of searches on "Gallup" and "poll" for the latter half of 2009 (though the Rasmussen gain fades considerably if I search on an exact match of "Rasmussen poll").

Clay Shirky, an academic who writes extensively about the Internet, recently floated an idea he calls "algorithmic authority." He defines it as "the decision to regard as authoritative an unmanaged process of extracting value from diverse, untrustworthy sources," such as Google, Wikipedia or Twitter, "without any human standing beside the result saying 'Trust this because you trust me.'" One characteristic of that authority, Shirky writes, is "when people become aware not just of their own trust but of the trust of others."

The wisdom of crowds may be a lousy way to evaluate polling methods, but my pollster colleagues should take note: When it comes to deciding which polls to trust, the producers and consumers of polling data often follow very different paths. And that divergence may be growing.

Get us in your feed.
More Legacy Content
 
Comments
comments powered by Disqus