© National Journal Group Inc.
Wednesday, March 20, 2002

Gallup recently conducted a groundbreaking poll of almost 10,000 people in nine Islamic countries -- Indonesia, Iran, Jordan, Kuwait, Lebanon, Morocco, Pakistan, Saudi Arabia and Turkey. Respondents were asked about their values, opinions toward the United States and feelings about the Sept. 11 attacks. NationalJournal.com's Anne Wagner talked with Gallup's director of international research, Richard Burkholder, about the poll's findings and what Americans can learn from them.

In none of the societies did the majority of respondents condone the attacks, but there was also condemnation of the U.S. military response.

Q. Let's start out by talking about the poll in general. What were you hoping to find out when you designed the poll and began the project?
A. Our CEO came to me a few weeks after September 11. He said, "One of the remarkable things is that we really don't have a very good read on, what is sentiment in predominantly Islamic societies? We have a few polls here and there from a few of the more open states, but we really don't have any kind of a comprehensive read on sentiment in those societies." So we set ourselves to the task of trying to design a survey that would meet opinion where it was in people's own terms at the popular level; and that we could execute it in as broad a grouping of Islamic societies as possible; and that had questions that were worded in a way that was acceptable to respondents and not intrusive to respondents; and that had a structure in the questionnaire that would engage them and give them a sense that there weren't any right or wrong answers to the questions, but that we really were interested in learning their personal views.

That may sound like a minor point, but in some of the societies polling is a fairly common phenomenon -- places like Turkey, Pakistan and so on. In other societies, polling on sensitive political issues is really breaking new ground. So there was a lot of work that had to go into designing the questionnaire in a way that would execute properly. I would say that we spent about seven weeks drafting the questionnaire, pre-testing the questionnaire, working with question wordings and watching it out in the field to see what worked and what didn't work well until we got an instrument that we knew would work well with folks.

Q. Had you ever done polling in any of these nations before?
A. Market research, yes. Sensitive political opinion polling, no. My background at Gallup -- I've been there 18 years now -- I've been involved with dozens of international projects. But as far as interviewing on sociopolitical issues, on sensitive political issues, personally, no. The entities that we chose to do the data collection for us were all local research firms whose business typically is predominantly market research. So they are familiar with the issues of drawing a representative sample, a good sample design, covering urban areas and rural areas, and all of those things, but the application is usually in the field of market research. The techniques are precisely the same in terms of a good sample design and good questionnaire design. It's the content that differs.

Q. The part of this poll that made headlines was the finding that the majority of respondents who were polled said they didn't think that Arabs carried out the September 11 attacks. Can you go into detail about some of the things you found out?
A. That has drawn a lot of attention. I think one of the points that has been made in commenting about it, that I've seen folks comment on, is that the administration was fairly slow to come forward with a lot of detailed information that would substantiate what we believe objectively happened and therefore left the field at play to other forces within some of these countries who portrayed the events in a different way.

We did ask an open-ended question about who they felt was behind the attacks. We got a variety of responses to that. Some of them said it was terrorists but wouldn't accept the characterization of them as Muslim or Arab terrorists. Some of them did refer to al Qaida specifically or [Osama] bin Laden's groups. There were even some fairly far-fetched theories with regard to conspiracy theories -- that it was either done by Israeli agents or even, the equally far-fetched one, that the U.S. itself might have staged the attack as some kind of a pretext. Those were minority views.

But that did get a lot of play. Again, that was late in the interview, and a lot of what we were trying to accomplish was to try and understand their own value system and what things were important -- what issues they had paid attention to -- and also to get their sense of reaction. It has to be said, in balancing that, that in none of the societies did we find that the majority of respondents condoned the attacks or regarded them as largely or totally morally admissible. But by the same token, there was also condemnation of the U.S. response to the attacks, in terms of the action the U.S. took militarily in Afghanistan. I think one of the threads that ties those things together is, given the fact that they were reluctant to accept the West's definition of who was behind the attacks, it made them of course much more reluctant to accept the U.S.'s military response to the attacks.

Q. What were some of the other political opinions and feelings that you found out through this poll?
A. A lot of the interesting things we asked had to do with attitudes towards the West in general, towards the West value system, towards the United States specifically, and also what sorts of attributes they associated with Western nations generally.

I think that in general, if you ask people in these societies, "What do you most admire about the West?" you find they give us great credit for scientific expertise, technology. They give us high credit, typically, for having political systems that respect human values and human rights and social freedom. Where they give us low marks -- and I'm talking about the West collectively and not the U.S. specifically -- if you ask them what they most dislike or most resent about the influence of the West, they will talk a lot about social aspects, about what is seen as morally decadent lifestyles with regard to gender relations, sexual relations, with regard to dress and what they see through common exported entertainment and things.

They also will tend to associate a certain amount of arrogance with us. They tend to think that we have negative attitudes toward them. There is a lot of resentment of the fact that they are not valued, or they believe that the West does not value their own culture and their own societies.

We read them a list of about 12 or 14 adjectives and asked them, without pointing toward any specific countries, which countries -- if any -- they thought those adjectives applied to. We told them that it could apply toward one or two countries, or just a single country, or all of the countries, and so on. [In] regards to the U.S., they thought we were scientifically and technologically advanced. Large majorities in every country said that. But when you get to some of the other positive attributes, very few people thought of the U.S. -- and I'm talking about fewer than one in five in any of these countries -- as being a trustworthy country, as being a place that was peaceful to live in.

When you get to the negative attributes, large majorities in most of the countries associated the U.S. with such adjectives as "aggressive," "conceited," "easily provoked," "ruthless," as a country which adopted biased policies in world affairs. Many in most of the countries thought of us as arrogant as well. That was the downside of their attitudes toward us.

Incidentally, we also asked them about their perceptions of their own societies. In general, they gave themselves the highest marks with regard to things such as being open-minded toward Western culture generally. They gave themselves relatively low marks with regard to whether or not their cultures had a promising technological and scientific future and whether or not their own cultures and societies were ones in which people were free to control their own lives and their futures. So they were critical of some aspects of their own societies as well as of the West. I think that has perhaps gone unreported.

What it shows is a balance, not only in the questionnaire itself but also in the views that were expressed to us.

They give us great credit for scientific expertise, technology. They give us high credit for having political systems that respect human values and human rights and social freedom.

Q. Can you tell me a little about the methodology of the poll -- how you conducted it and how it differed from a Gallup poll in the United States?
A. It's essentially identical to the way a Gallup poll in the U.S. would have been conducted, say, 15 or 20 years ago. When I first came to Gallup, we were still doing the bulk of our sensitive political work, and certainly our pre-election work, using face-to-face, in-home interviews of a national probability sample. We switched over in the late '80s, even on our final election call -- 1988 was the first time I think we did it totally by telephone -- to telephone interviewing. Telephone ownership levels had become so high that that was an equally acceptable manner.

Obviously, telephone interviewing is not an appropriate methodology in societies where telephone ownership is relatively low. There are some other concerns with it as well -- you're contacting folks in a way that is completely anonymous and from a distance.

The sample design is a national probability sample of rural and urban households. We first stratified the population of the country by known sampling parameters such as census data. Then we selected a wide variety of spots around the country on that basis. Within those spots, we selected households on a random basis. Within households, selected adults using a system called a Kish grid where we randomize the selection of the actual adults. Everyone in the entire society should have a known chance of inclusion in the survey. Unless we are slightly undersampling rural areas because of their remoteness and then weighting them up, which we can do, everyone has an equal chance of inclusion.

Q. How long did it take to conduct this poll?
A. An individual interview would take almost an hour. It was a very exhaustive interview that goes into a lot of things. The poll itself, in the field, we went into the field in early December and finished the field work in late January. That's not unusual. For a survey of 1,000 hour-long interviews, where you are covering remote rural areas, that's about a typical field period....

Q. Did you find that people were willing to talk to you?
A. Actually, the cooperation rates on average are better, I would say, than the norm for some forms of telephone interviews in the States. We had very few refusals -- no higher than about one in four or one in five in the most extreme cases. So yes.

I think there is a process where people aren't used to being interviewed, where you have to engage them, as I said, and ask a lot of questions first about their general interests in life, the importance of things to them -- family, reading behavior, media habits, lots of things -- so that they do get a sense that there aren't any right or wrong answers and you are sincerely interested in learning about them. But in general, once people had that rapport established with the interviewer, I watched an awful lot of interviewing in the field, and the rapport was quite good and the cooperation rates were very high.

Q. We've read that some of the poll's questions couldn't be asked in some of the countries because of censorship. Do you think that these restrictions affected the poll's findings at all?
A. Well, obviously to the extent that we didn't ask the question it wouldn't affect that. But there were four or five questions, perhaps, out of an hour-long interview, that were just either too direct or too sensitive for us to feel we would have gotten a good measure on them.

So in the process of clearing this with authorities and also judging it ourselves and what worked in the field, we might get away with asking a question like, "Who do you think was behind the attacks?" But pre-listing who those people were wouldn't have worked. In other cases, it's a question of wording the question in a way that would be not offensive to the respondents.

Let me give you one example that I've given before. If you wanted to ask, "Do you think the September 11 attacks were in keeping with or in contradiction to, the teaching of Islam?" That's a little bit too direct because it puts the respondents into giving us sort of a theological interpretation and also [asks them to comment] directly on their view of what Islam does and doesn't mean to them. But if you said, "I'm going to read you a list of things that can happen, of things that people may do in life, and I want you to tell me whether you think they are completely justifiable" -- this was a five-point scale -- all the way down to "completely unjustifiable," and then you list into that series a whole bunch of things -- divorce, forcing opinions onto other people, exceeding the speed limit and so on, and one of those items is these attacks, they will give you, in the course of that context, a sense of what they think the moral admissibility of those actions is or is not. Within the context of an Islamic society, to say whether or not they believe something to be morally admissible or not, that moral compass essentially comes from their understanding of Islam, with the exception perhaps of the minority in Lebanon who would be Christian.

Q. Were most of the people you interviewed for this poll Muslim?
A. In every society, the majority were Muslim. There are significant populations of Muslims in countries where we did not interview because they were not the majority population within their country. A good example would be India, which has in excess of 125 million Muslims, something like that. But it's not a predominantly Muslim society, and therefore, some of the questions about the impact of the West value system on your traditional values would not have applied in the same way. The largest case where there was a significant non-Muslim minority would be Lebanon, where something just over 40 percent of those that we interviewed would have been non-Muslims. And that's an interesting case in another way because the Lebanese data in some ways are sort of bimodal on some things such as perceptions of whether or not Arabs were behind the attacks. Christians were much more willing to openly say, "Yes, we believe that was the case." When you asked that direct question, that was less likely to be the case among Muslims in Lebanon, who presumably have access to the same media.

Q. Within the different countries, was there a lot of variation in opinions?
A. This is a tremendously disparate and diverse set of societies. We went from Morocco... all the way to Indonesia, which is the largest Muslim society in the world. And there are societies within this grouping, such as Turkey, which are largely integrated into Western economic and political structures -- Turkey is a member of NATO -- running the gamut all the way to an Iran or a Saudi Arabia, which is an extremely conservative, traditional, religious-oriented society and nothing like the secular society that Turkey is. So, yes, there were differences.

One of the more interesting things, though, was the degree to which you couldn't always perfectly predict or assume you understood a society's feelings just on the basis of your presumptions about them. There were some misgivings, for instance, in Turkey, about the impact of Western values on their culture. There is this sense that they are less attached to religion, obviously, than their forebears and their forefathers were. And on other questions, it's not necessarily the case, for example, that the attitudes in Iran were the most extreme. They weren't. It varied issue by issue.

There's a lot to be gained not only looking at these data in the aggregate but also at the opinions within specific societies.

Q. What were some of the more interesting cases of that that you saw?
A. One that's drawn some attention is the fact that perhaps Kuwait is a more conservative society and a more traditional society than some people have assumed. People who haven't traveled there think of it as a very secular and pro-Western society, and I think that's an oversimplification. We asked a lot of values questions about people's attitudes toward various forms of social behavior, whether it was permissible to marry against one's parents' wishes, whether polygamy was permissible, lots and lots of things. And Kuwait tends to be toward the conservative end of the scale on those questions. So that's drawn some interest. I talked about Turkey a second ago and that the attitudes in Turkey, while they are on the pro-Western end of this continuum, are not on some issues extremely different from those of the other countries in this grouping.

Q. Some prominent Arabs -- including those at the Kuwaiti embassy and the former Pakistani ambassador to the United Nations -- have criticized the poll, saying it doesn't accurately reflect public opinion in their countries. What is your response to that?
A. The criticism that I have heard from the Kuwaitis is that we ought not to have interviewed people who were not formally holders of Kuwaiti citizenship. And I need to explain that a little bit. On official numbers, and these are the Kuwait government's own numbers, 58 percent of the population of Kuwait does not formally hold citizenship. They're called expatriates. Many of those folks were born in Kuwait, may have lived their whole lives there. But they're not descended from families who were residents of Kuwait several generations back. The Pan-Arab Research Center, which itself is based in Kuwait and which conducted the interviewing, has been interviewing there for over a quarter of a century. It's the norm in Kuwait to interview both expatriates and Kuwaiti nationals in doing the survey. Otherwise, you exclude 58 percent of the adult population of the society. So we didn't do anything different in that sense.

Now, we know just from demographic questions we asked whether a specific respondent does hold citizenship or not. And we have done internal analysis to compare the views of those who have citizenship and those who are considered expatriates. And they're not, across the board, all that different, frankly. Sometimes the differences cut in one direction, sometimes they cut in a different direction. If you look at the series, for example, of adjectives that I just read to you about the United States -- whether the U.S. is perceived to be arrogant or aggressive or conceited or ruthless -- on every one of those issues, the differences are statistically insignificant. It's true that Kuwaiti nationals will give a slightly higher overall appraisal of favorability to the U.S. But on a five-point scale, it's a 3.0, and with expats it's a 2.6. So it's really not a significant difference at all. And on some other critical issues, some of the more controversial issues, the views really aren't easily discriminated at all.

So, to be honest, I think some of the criticism has been directed in a way which I think doesn't take proper account of the fact that this is a rigorous sample design of the entire population. It was done in the way it's been done for a quarter-century there, and that was done in-home, in private, in Arabic. So we have confidence in the data that we have from them.

There were some misgivings in Turkey about the impact of Western values on their culture. And Kuwait is a more conservative and traditional society than some people have assumed.

Q. What about the criticism from countries like Pakistan?
A. I have seen no official criticism from the government in Pakistan. What I've often heard, frankly, is similar to what Tom Friedman wrote in the [New York] Times... which is, if anything, he's surprised that the attitudes aren't more negative than they are. I think you cannot legitimately criticize on a technical basis the way the opinion was gathered. There have been people who have argued with the substance of the opinions that were expressed, but, frankly, their argument is with the sample and with the people that we interviewed. We don't really take a position on what the findings ought to be in a specific society. Our job is to do the best and the most rigorous and most scientifically defensible measurement we can do.

Q. What questions would you use as follow-ups to this survey?
A. One of the things that a number of people, like former Ambassador [to the United Nations] Richard Holbrooke, have said is that this is a very important survey and it's something that needs to be tracked and continued over time. We deeply believe that. I think we need to measure attitudes towards the West, perceptions of the West. There also were some questions about what the West could do to improve relations with Islam, and I think we need to measure those and track those over time.

Some of the leading things that came up in response to that: We asked people what steps the West could [take] to improve the quality of its relations with the Islamic world. We had at least one in ten say the West ought to moderate its attitudes toward Arabs and show less prejudice and discrimination toward them. We had a lot of folks tell us that it ought to demonstrate more respect and more understanding of the status of both these societies in general and of Islam specifically. The third most common mention in that category was it ought to increase economic support and promote investment in the region to help improve the economic lot of folks there. And another large group said it ought to adopt a more favorable stance in foreign policy, particularly in the area of Palestine and Palestinian-Israeli relations.

I think those are all things worth monitoring. And I think it's also important... we issued a Gallup poll conducted in the United States on U.S. attitudes towards Islamic societies in some of these countries specifically. I think that we need more information and more dialogue in general over time between these groupings because it's been sadly neglected.

I think it's important to understand that we're not always perceived the way we think we're perceived.

Q. Is this something you're planning to do a lot of follow-up on?
A. Yes, we are. Our CEO has already said that we are committed to tracking this on an ongoing basis. It represents a significant financial investment by us to do that, but we think it's important enough that we have underwritten this ourselves and we will continue to do so.

Q. Finally, I'd like to ask how U.S. legislators and officials can use the information from this poll.
A. I think if you're talking about Congress, I think it's important to understand that we're not always perceived the way we think we're perceived.... I want to stress that we would have gotten some of this had we interviewed in Latin America, or in Asia, or even in Europe. I think there's a sort of American sense to some extent that if we know what we think we know about ourselves, it must be readily apparent to other people as well. And I think it's very important to see how we're perceived through others' eyes and what sorts of things lead us to be perceived that way.

I'll give you one example. When we get the criticisms of cultural influences -- and they like our entertainment products and they like our music and films and so on, but there is some criticism that the lifestyles portrayed in them [are] not particularly moral or is in some ways decadent. I think when our own conservative groups look at that and see our entertainment products that have that character to them, they say, "This is unfair because this doesn't really represent mainstream America." I think what's perhaps doubly troublesome in this case is that, to the extent that that is their image of America, what's seen through our entertainment exports, they may find it distasteful but they may not be adequately aware that it does not necessarily represent the normal life of normal American families in Middle America every day.

What I would recommend to legislators, to Congress, is to take a good look at how you are seen around the world and what is the basis for that so that when you communicate, you can communicate in a way which addresses misperceptions and gets to trying to smooth over those perceptions which may have some legitimacy to them.

One final thing I would say is that one of the criticisms about the West and about the U.S. is that we are not particularly religious, that we don't have a sense of our own values and a commitment to religious values. If you know the data objectively, the U.S. is -- particularly compared to Western Europe -- actually a very religiously observant society in relative terms. That aspect of American culture is not something that they would necessarily see or be aware of. That's a case, perhaps, of misperception and something that can be spoken to directly.

[ Insider Interview Archives ]

Need A Reprint Of This Article?
National Journal Group offers both print and electronic reprint services, as well as permissions for academic use, photocopying and republication. Click here to order, or call us at 877-394-7350.