Researchers debate effectiveness of exit poll questionnaires

It was a simple multiple-choice question. The reaction to the answers has been anything but. On Election Day, exit pollsters posed a string of questions to voters after they cast their ballots, including the following: "Which one issue mattered most in how you voted today? Taxes, education, Iraq, terrorism, economy/jobs, moral values, health care." "Moral values" was chosen by 22%, with "economy/jobs" at 20% and "terrorism" at 19%. In the aftermath of President Bush's victory, analysts of all political persuasions seized on the results of that single question, with some arguing that it reflected the nation's conservative shift and a widespread rejection of gay rights, abortion, and indecency in entertainment. Others contended that far too much was being extrapolated from just 22% choosing "moral values," two words that were too broad to provide specifics let alone worthy analysis. Soon after the elections, the Pew Research Center asked a similar question--without offering a laundry list. In that survey, "moral values" was chosen by only 9% of respondents, behind "Iraq" at 27% and "the economy" at 14%. Just over four in 10 of those who picked "moral values" mentioned issues such as gay marriage and abortion, but others cited qualities such as religion, helping the poor, and candidates' honesty and strength of leadership. Some public opinion researchers such as Jon Krosnick of Stanford University question whether a term such as "moral values" should be included on a list of specific issues such as health care, education, and the economy. "The responses mix policies that have clear meanings" and a broad term like "moral values," Krosnick said, adding that "moral values" could refer to many different concerns. Republican pollster Whit Ayres said the role of moral values in the election goes far beyond the debate over one poll question. "While this was a national security election, any way you read the data, moral and cultural values were a very important factor in the president's vote," Ayres said. Words and how they are used in poll questions can elicit a strong response. Or, as in some cases, prove to be highly suggestive. Consider this question from the late 1930s: "Would you vote for a woman for president if she were qualified in every other respect?" When asked that question, 33% said yes, and 64% said no--almost 20 years after the 19th Amendment to the Constitution gave women the right to vote in the first place. In a more recent example, the Associated Press in mid November asked the following question about abortion and the Supreme Court: "The 1973 Supreme Court ruling called Roe v. Wade made abortion in the first three months of pregnancy legal. Do you think President Bush should nominate Supreme Court justices who would uphold the Roe v. Wade decision or nominate justices who would overturn the Roe v. Wade decision?" Six in 10, or 59%, said they preferred justices who would uphold the court's decision. Pundits criticized the question, specifically the phrase "in the first three months of pregnancy." The court ruling actually does more than make abortion legal in the first three months of a pregnancy--abortion would be legal further into a pregnancy if circumstances called for it. In early December, the AP asked a slightly amended poll question, saying simply that Roe v. Wade "made abortion legal" and removing the reference to the first three months. In response to the new question, 57% said they favored justices who would uphold the court's landmark ruling--essentially no change. Public opinion researchers offered various theories on the relatively insignificant change in responses in this instance. Robert Shapiro of Columbia University suggested such shifts in question wording may have little or no effect on strongly held beliefs. "On longstanding issues that have been debated for a long time such as abortion and capital punishment," Shapiro said, "small changes in question wording tend to make little difference unless it substantially changes the substance of the question."