ORLANDO, Fla. — During the 2008 presidential campaign, analysts warned that then-Sen. Barack Obama's lead might be overstated by the polls as a result of the so-called "Bradley Effect" — the social-desirability bias that occurs when white voters lie to pollsters about their intention to vote for a minority candidate. But a new research paper points to another element of such bias: In 2008, poll respondents were more likely to say they would vote for Obama if the person conducting the interview was African-American.
The relationship was minor but significant, with respondents contacted by black interviewers found to be 2 percentage points more likely to say they would vote for Obama. The difference was more pronounced earlier in 2008, and it informs why Obama ran stronger in polls by some survey houses and weaker in others, particularly in the spring of 2008.
The paper — authored by Stanford University Ph.D. candidate Nuri Kim — was presented here at the second full day of the American Association for Public Opinion Research's annual conference. Kim analyzed surveys from ABC News/Washington Post, CBS News, and Gallup.
Each organization has already reported on what are called race-of-interviewer effects in their respective preelection polling in 2008, Kim said. ABC News/Washington Post and Gallup reported that results they examined showed no effect, while CBS News said its polling before the parties' conventions in late summer showed some differences among white respondents.
But Kim's paper found larger differences, she told pollsters and other survey researchers on Friday afternoon. In March, April, and May of 2008, she found all three survey houses reported significant differences in results depending on the race of the interviewer.
In ABC News/Washington Post polling, respondents were 3.9 percent more likely to say they would vote for Obama if they were contacted by a black interviewer. CBS News reported an even greater difference (4.8 percent), while Gallup reported a smaller difference (1.3 percent). (The discrepancy in Gallup's polling is statistically significant due to the large pool of respondents surveyed in their daily tracking poll.)
College-educated voters were less likely to be influenced by race-of-interviewer effects compared to noncollege voters, Kim reported. A larger-than-normal discrepancy was also reported among self-identified Democrats during the spring; then-Sen. Hillary Rodham Clinton was still a candidate at that point.
Part of the problem relates to recruitment of survey respondents. Studies have shown that people comply with surveys from people that they like, and "the strongest predictor of liking is similarity," Kim said.
Indeed, her analysis of the data from those three survey houses found that white interviewers were more likely to recruit white respondents, and black interviewers were more likely to recruit blacks. Americans are quite adept at discerning the race of a telephone interviewer, Kim said.
Kim also reported a discrepancy in the number of black interviewers among the three survey houses. ABC News/Washington Post had the highest percentage of black interviewers, with 37.5 percent; Kim's study only looked at white and black interviewers, so these percentages exclude interviewers who were Hispanic, Asian, or of other ethnicity. About a quarter of CBS News's interviewers were black, and only 1-in-10 of Gallup's interviewers were black.
Accordingly, during the spring, Obama ran stronger in ABC News/Washington Post and CBS News polls than he did in surveys conducted by Gallup.
If the percentage of black interviewers is producing either too many or too few black respondents through recruitment biases, weighting should help correct that imbalance. But Kim says that weighting does not correct the problem entirely.
"Weighting does mitigate interviewer effects," Kim said, "but it's not completely eliminating them."
Kim looked at 36 total surveys conducted by those organizations between March and November of 2008, looking only at registered voters interviewed by white or black interviewers. Kim worked on the project alongside Stanford professor Jon Krosnick, who is a frequent presenter at the AAPOR conference.
Kim's study did show that in the time period between the parties' conventions and the election, the difference in vote choice expressed by respondents with white and black interviewers at those three survey houses virtually disappeared by Election Day.