How much does the pollster concept of "measurement error" inform the debate about counting the so-called popular vote cast in the Democratic primaries and caucuses this year? More than you may think.
Measurement error is the term survey researchers use to describe what happens when a respondent's answer (or an interviewers record of that answer) does not accurately describe the underlying opinion or characteristic that the pollster was hoping to measure. Suppose the respondent does not hear or understand the question, or feels too embarrassed to share the true answer, or perhaps the interviewer checks the wrong box. Any of these problems can lead to inaccurate measurements.
Obama leads the popular vote in this year's Democratic primaries and caucuses, but several vote-counting issues make measuring that lead difficult.
Although he did not use the term, UCLA history professor Theodore Porter invoked the concept of measurement error in a Washington Post op-ed penned in the midst of the Florida recount battle eight years ago. Since the lead that George W. Bush held over Al Gore in Florida at the time hovered "around one one-hundredth of one percent," he argued that the race was "essentially a draw" in which "the number of votes lost due to errors in Florida is much greater than the spread between Bush and Gore."
The final report of the Florida Ballots Project of the National Opinion Research Center (NORC), proves Porter's point. Although Bush led on the final certified count by just 537 votes, 61,190 Floridians invalidated their ballots with "overvotes" (usually by marking more than one candidate). According to the NORC report, these ballots included 5,277 in Palm Beach where voters punched out chads for both Gore and Pat Buchanan on the now infamous butterfly ballot. The report also found thousands of officially uncounted ballots where voters incorrectly circled or wrote in a candidate's name on an optical scan ballot.
The point here is not to reopen the debate about which ballots should have been legally counted or recounted, but to point out that the final result fell well within the margin of measurement error of Florida's vote counting system. Whatever the legal consequences, we know that tens of thousands of Floridians went to the polls eight years ago intending to express a preference that was not measured accurately by the counting system.
Fast forward to the notion of the "popular votes" cast so far in the 2008 Democratic primaries and we confront much bigger measurement issues. If we check the RealClearPolitics tally of the votes cast in the 41 "official" contests recognized by the Democratic National Committee (DNC) to determine delegates to the Democratic convention that released popular vote counts, Barack Obama currently leads by 717,086 votes, or a margin of 2.6% of the more than 13 million votes cast.
That part is easy. Unfortunately, setting the run of the mill counting errors that were on display in Florida in 2000, we confront several bigger counting issues:
• Four states (Iowa, Nevada, Maine and Washington) held caucuses but did not report a head count of the preferences of participants. They reported only the number of delegates elected to subsequent state conventions. How do we count the preferences of the voters that participated?
• RealClearPolitics publishes a count that includes popular vote estimates for the four caucus states. That count adds roughly 110,000 votes to Obama's total, presumably imputed from each candidate's share of the state delegates to the estimated turnout totals reported for each state (Obama-friendly DailyKos provides more detail on an estimate that has roughly the same bottom line). Of course, these estimates are inherently imprecise given the switching of preferences on the second round of caucus voting.
• In Washington, the turnout for the non-binding primary was more than double that of the officially recognized caucus that chose the convention delegates. Using the primary rather than the caucus cuts Obama's lead by roughly 50,000 votes.
Which brings us back, of course, to Florida or Michigan, the states whose primaries have not been recognized as legitimate by the DNC. Setting aside, for a moment, the politically charged (and obviously ironic) question of whether those votes should be counted, and consider the measurement problems in Michigan, where the ballot featured only Clinton and Kucinich. Jay Cost's provides a concise summary:
[We] could (a) give Obama the "unaffiliated" vote, (b) not give Obama the "unaffiliated" vote, or (c) reallocate the vote based upon whom voters claimed in the exit poll they would support if all candidates had been on the ballot.
For now, at least, Barack Obama's popular vote advantage exceeds the margin of measurement error. He leads on all six counts posted by RealClearPolitics, including those that include unadjusted votes in Florida and Michigan. But before we start to ponder where where these counts may end up after all of the contests are complete, we need to consider a bigger issue, which is less about an accuracy than about what sort of counting is most appropriate. Or, to use the language of pollsters, the issue is not just about the accuracy of the vote count, but its validity.
I will take that question up next week.
Want to stay ahead of the curve? Sign up for National Journal’s AM & PM Must Reads. News and analysis to ensure you don’t miss a thing.
Join the Discussion