Like President George W. Bush before him, President Obama has taken great pains to insist that America's conflict is with extremism and terrorists, not the Islamic religion itself.

But the inflammatory controversy over the proposed mosque and community center near Ground Zero in Lower Manhattan suggests that a substantial portion of America -- particularly Red America -- simply does not agree. That could make it much tougher for Obama, or any of his successors, to improve America's image in the Muslim world -- a project that recent international surveys suggest is already faltering after Obama's election generated an initial surge of good will.

It is simplistic and misguided to assume that everyone who opposes the mosque is hostile to Islam or views the religion as inherently threatening to American security. Polls show that many Americans simultaneously recognize the sponsors' right to locate their mosque wherever they choose and wish they would choose not to site it on such troubled ground.

In a poll released last week, a

But the debate has also unleashed a harsher and more sweeping skepticism about Islam, mostly among conservatives and Republicans. Not for the first time, former House Speaker Newt Gingrich has been the most bombastic. He's likened the mosque to an attempt by Nazis "to put up a sign next to the Holocaust museum" and insisted (without any evidence) that the imam behind the project is "trying to make a case about supremacy" by proving that Muslims "can build a mosque right next to a place where 3,000 Americans were killed by radical Islamists." Rush Limbaugh similarly insists the mosque is intended as "a victory monument" for the 9/11 attack. And conservative columnist Charles Krauthammer has warned that the mosque could one day house a radical anti-American cleric like the one who influenced the Fort Hood gunman.

That's a striking accusation. The project's sponsor, Imam Feisal Abdul Rauf, may have been too equivocal after 9/11 when he condemned the attacks but also described American policies as "an accessory to the crime." But far from promoting extremism, Abdul Rauf has consistently preached that Islam can peacefully coexist with other faiths. "He has stood for a more tolerant approach to Islam," said Richard Haass, director of policy planning in Bush's State Department. To suggest that Abdul Rauf could turn and harbor radicals isn't far from implying that any Muslim might do so.

Some prominent GOP thinkers have warned against painting Islam and American Muslims with such a broad brush. But these voices (primarily former Bush aides) have been eclipsed by the opponents. Although the rest of the GOP's emerging class of potential 2012 contenders haven't matched Gingrich's rhetorical excesses, Sarah Palin, Mitt Romney, Mike Huckabee, and Tim Pawlenty have all opposed building the mosque at its current site.

There's clearly an audience for that position: Polls have consistently shown that a majority of Americans, including not only Republicans but also independents, oppose locating the mosque near Ground Zero. But surveys in the controversy's wake have also shown a surprisingly broad receptivity to much harsher assessments of Islam, particularly among Republicans and conservatives.

In a national Time magazine poll released last week, just under half of all Americans agreed that Islam is more likely than other faiths to promote violence against nonbelievers. But that number rose to 70 percent among Republicans and nearly three-fourths among conservatives. Fully 55 percent of all Americans said they believed that most U.S. Muslims are patriotic; but only 42 percent of Republicans and 38 percent of conservatives agreed. Perhaps most strikingly, 43 percent of conservatives and a 48 percent plurality of Republicans said Muslims should not be allowed to run for president. Only about one-fourth of Democrats and independents agreed.

In American politics, constituencies always find champions. It is inevitable that some prominent conservatives will articulate the suspicion of Muslims on the right evident in surveys like this (especially with Bush no longer around to suppress such arguments in his party). Obama has concluded, as Bush did, that one way to discourage extremism is to refute the terrorist insistence that America and Islam are locked into an inexorable clash of civilizations. The sobering message of the mosque controversy is that the Muslim world will lastingly receive a competing message from a substantial American constituency who believes that they are.

Want to stay ahead of the curve? Sign up for National Journal’s AM & PM Must Reads. News and analysis to ensure you don’t miss a thing.