The fundamental, founding principle of polling is the probability sample -- the notion that each member of the population has an equal chance of being selected to participate in the survey. But what if most public polls in many of this cycle's most competitive Senate races were a priori disregarding roughly 40 percent of the population?
A new report Friday from the Center for Disease Control's National Center for Health Statistics underscores this problem. The study estimates of the percentage of voting-age adults living in households without a landline telephone by state. The latest estimates from the second half of 2011 are that roughly one-in-three adults age 18 and older has only a wireless telephone, but the new report shows even higher percentages across some of the states that could decide which party controls the Senate and the White House next year. That calls into question many of the polls that are released publicly conducted using automated phone interviews.
Interactive-voice response polls, sometimes called "robopolls," use a recorded voice to ask questions, with respondents indicating their answers by pushing a button on a touch-tone keypad or having their vocal response interpreted by voice-recognition software. It is illegal for automated dialers to call cell phones, which makes it illegal for IVR polls to reach respondents on cell phones (and more expensive for live-interviewer firms who conduct polls using computer-assisted telephone interviewing technology).
Arizona is home to one of the nation's hottest Senate races. Arizonans are also significantly more cell-only than the overall population, with 38.2 percent of those over 18 living in wireless-only households during 2011, according to CDC estimates. That was a sharp increase from 2010, when 33.2 percent of the adult population was wireless-only -- suggesting that the 2012 cell-only share of the population likely exceeds 40 percent.
For most of the year, it was thought that Republican Rep. Jeff Flake would defeat the Democratic nominee, former Solicitor General Richard Carmona, but the Senate race has closed over the past few months. Both sides released internal polls earlier this week, ranging from a 4-point Carmona lead in polling for the Democratic Senatorial Campaign Committee, to a 6-point lead for Flake in polls for his campaign and the National Republican Senatorial Committee's independent-expenditure arm. (Pollsters for the DSCC and NRSC polls contacted some respondents via cell phone, Hotline On Call confirmed, though these percentages fell short of the cell-only adult population estimate. A pollster for the Flake campaign did not return a phone message on Friday afternoon.)
Observers seeking some independent data on the race would not find much in the way of reliable polling. The Democratic automated polling firm Public Policy Polling found Carmona ahead by 2 points earlier this month
, but PPP doesn't call cell phones, meaning roughly 2-in-5 eligible voters have no chance of being contacted by their computers.
PPP does weight their results by age, race and gender to account for these shortcomings, in addition to other sampling quirks. According to national trends, we can expect that not calling cell phones would require younger voters, minorities and males (who are less likely to own landline phones) to be weighted up, while older voters, whites and females (more likely to live in households with landlines) would be weighted down.
"Weighting usually takes care of most of the problems associated with not calling cell phones," said PPP director Tom Jensen.
But as the percentage of cell-only Americans continues to rise, that puts additional strain on these weights and could bias results. Jensen concedes that his firm "may get burned this election cycle" because of this noncoverage bias, though he expressed confidence in their methodology.
The November election will go a long way toward determining the importance of calling cell phones for political surveys. But what is clear now is that the effects are likely to vary across the states. Friday's new report shows that, while some heavily-polled states have high cell-only populations, others have been more likely to retain landlines.
Here's a look at the latest estimates from many of the battleground states at the presidential, Senate and gubernatorial levels this year:
-- Arizona: 38.2 percent (up from 33.2 percent in 2010).
-- Colorado: 38.7 percent (33.2 percent)
-- Connecticut: 18.7 percent (15.3 percent)
-- Florida: 34.4 percent (30 percent)
-- Hawaii: 26 percent (23.1 percent)
-- Indiana: 32.8 percent (28.3 percent)
-- Maine: 31.6 percent (25.7 percent)
-- Massachusetts: 21.3 percent (17.3 percent)
-- Michigan: 35.8 percent (30.8 percent)
-- Missouri: 32.5 percent (25.8 percent)
-- Nevada: 34.7 percent (27.9 percent)
-- New Hampshire: 23.6 percent (17.3 percent)
-- New Mexico: 36 percent (29.9 percent)
-- North Carolina: 32.8 percent (27.1 percent)
-- North Dakota: 41.6 percent (38.1 percent)
-- Ohio: 33.4 percent (28 percent)
-- Virginia: 26.6 percent (22.6 percent)
-- Washington: 33.8 percent (30.2 percent)
-- Wisconsin: 33.8 percent (29 percent)
Not all states are created equal. North Dakota law prohibits robopolls entirely, but some live-caller pollsters in that state's closely watched Senate race might also opt to exclude cell phones to keep costs down. Doing so -- and ignoring the nearly 42 percent of voting-age adults who live in cell-phone-only households -- would make their polls subject to greater noncoverage bias than a poll in Connecticut, where fewer than 19 percent are cell-only.
Furthermore, the rates of the changes within each state show that methods that are found to produce accurate surveys in 2012 may not repeat that accuracy in 2014 or 2016. If a pollster who ignores or calls too few cell phones is accurate in this election, the rapid changes might catch up with them in the next election.
As Americans become even less dependent on landline phones, automated and live-caller landline-only polls are likely to become even more anachronistic. According to Friday's report, that has already happened in many key states this election cycle. Polls in these states that undersample cell phones -- or ignore them entirely -- should be viewed with increased skepticism.