When all the votes were tallied in Pennsylvania's Democratic Senate primary last week, challenger Joe Sestak defeated incumbent Sen. Arlen Specter by an 8 percentage point margin, 53.9 percent to 46.1 percent.
The final round of pre-election polls had suggested a closer race: The RealClearPolitics average of six final polls showed Sestak leading by slightly more than 3 points (44.2 percent to 41.0 percent). The Pollster.com trend estimate, which gave more weight to earlier surveys, had Sestak leading by less than a percentage point (43.1 percent to 42.5 percent).
Compared to the 2008 New Hampshire polling misfire, the understatement of Sestak's ultimate margin may seem minor, but the polling averages missed the margin by 5 to 7 percentage points in Pennsylvania. In New Hampshire, the final polling averages were off by roughly 9 percentage points. Aside from the difference in magnitude, why no hue and cry about Pennsylvania?
First, all of the polls picked up the huge trend that had propelled Sestak from double-digit deficits in March to dead even or better just before the election. The "more sensitive" trend line on our chart at Pollster.com showed Sestak gaining on Specter by roughly a half percentage point a day during May. Moreover, most polls stopped interviewing on the Saturday before the election, and their final snapshots reflected 3 to 5 days of calling. So if the trend to Sestak continued or accelerated over the final weekend, which seems highly likely, no one was surprised.
Second, even though most of the margins were not quite statistically significant, three of the four final polls showed Sestak with more support than Specter and one -- conducted by Suffolk University -- had Sestak's ultimate margin exactly right. So again, unlike New Hampshire 2008, most polls had the ultimate winner leading, nominally.
Third, there is the issue of how to interpret the large number of undecided respondents, ranging from 9 to 16 percent on the final round of polls. Did undecideds "break" disproportionately for Sestak? I obtained data on the undecided voters interviewed in the final week by Quinnipiac University and Muhlenberg University. Fewer than one-in-five had unfavorable impressions of Specter, but most were unaware of Sestak and roughly half were either unfamiliar with or neutral about Specter. Thus, most classified as undecided were probably non-voters, but a disproportionate share of those who did vote may have decided to support Sestak.
But fourth is another poorly understood issue that may have contributed to the understatement of Sestak's ultimate margin. I raised it nearly a year ago in a column that looked at, among other things, whether Specter was vulnerable to the coming challenge from Sestak. It focused mostly on the difficulty pollsters have interviewing and modeling the likely electorate for the low turnouts that are typical in primary elections.
Last week's Pennsylvania primary was no exception. According to the Pennsylvania Department of State, just over a million votes were cast in the Senate primary, a turnout that amounts to roughly 12 percent of Pennsylvania's registered voters and roughly 11 percent of its eligible voters.
It is difficult, to put it mildly, to anticipate and survey that likely electorate with precision. The tools available to pollsters are blunt instruments. The best we can do is create a rough approximation of the potential electorate and determine whether the horse-race numbers will differ significantly in higher or lower turnout scenarios.
It appears that in Pennsylvania, pollsters got a more accurate picture of Sestak's support as they narrowed their universe to the most likely electorate.
A survey conducted in early May by Franklin and Marshall University, for example, showed Specter leading by 9 points (38 percent to 29 percent) among all registered Democrats. But they found Sestak nominally ahead (38 percent to 36 percent) among a much smaller subgroup of likely primary voters.
And the tracking survey conducted by Muhlenberg University for the (Allentown) Morning Call showed the race finishing in a dead heat (44 percent to 44 percent) among all likely voters. However, when poll director Chris Borick narrowed the universe at my request to the subgroup who said they were "definitely going to vote" in the primary election, Sestak had a bigger advantage during the final week of tracking. For the last four days, Sestak led by 5 points (46 percent to 41 percent) among the definite voters.
I also spoke to two more pollsters who did not want to speak for the record, but who nonetheless saw a virtually identical pattern late in the campaign.
It would be foolish to draw broad lessons about how to model likely electorates in primary campaigns based on this admittedly limited data, and I am not recommending that pollsters filter for just "definite" voters in future primary polls.
But the relatively consistent pattern in this case suggests that in low-turnout primaries, especially where vote preferences show consistent differences at various levels of turnout, pollsters would do better to report a range of results -- showing how vote preferences differ as the likely voter universe narrows -- than a single set of horse-race numbers.
DON'T MISS TODAY'S TOP STORIES