Can You Trust the Polling on Proposition 8?

Do voters lie to pollsters when asked their opinion on banning same-sex marriage? That seems to be the question in California, where a survey last week found voters rejecting a proposed statewide ban on gay marriage by a decisive 17-point margin. Conservatives have countered with a study claiming that voters -- driven by "political correctness" -- substantially underreport their support for banning gay marriage to pollsters.

But a careful analysis of polling data on marriage amendments and election results from previous races indicates that if any such reluctance exists with regard to same-sex marriage initiatives, it is small -- about two points on average since 1998. In 2006 it was effectively zero. In other words, if the election were held today, polling trends suggest that California voters would reject the marriage ban -- known as Proposition 8 -- by a healthy margin.

The Field Poll in question was conducted September 5-14 and found likely voters opposed to the ban by 55% to 38%, with 7% undecided. The poll is the second in as many months indicating that a majority of the state's voters intends to vote no on Prop. 8. (If passed, the measure would reverse a May ruling by the California supreme court that gave same-sex couples the right to marry.)

The response by Prop. 8's sponsors? Not so fast, they say. The group ProtectMarriage.com released a study last week that compared preelection polling with actual Election Day results in 26 states that have voted on same-sex marriage initiatives since 1998. They contend that many survey respondents who supported the bans misreported their preferences to pollsters as undecided or even opposed, and conclude that support for these measures has been underestimated by an average of seven percentage points.

But my analysis of the ProtectMarriage.com data suggests that the study's methodology overstates the gap between polls and election results with calculations that implicitly allocate undecided survey respondents to the "oppose" category. Reanalyzing the data shows that the number of voters who are reluctant to share their true feelings about same-sex marriage is small -- and is certainly not on the rise. Since 1998, the gap between polled support for marriage bans among decided voters and Election Day results has averaged only 2.2 percentage points. In 2006, the gap declined to less than a point in the seven states holding initiatives for which data are available.

Though ProtectMarriage.com's methodology was flawed, the phenomenon of voters underreporting their biases -- often referred to as "the Bradley effect" -- does have a historical basis. It takes its name from Tom Bradley, the African-American former mayor of Los Angeles who staged an unsuccessful run for governor of California in 1982. Two Field Polls conducted shortly before the 1982 election found Bradley with a significant lead over his white Republican opponent, George Deukmejian. But ultimately, Bradley ended up losing by just one percentage point. Analysts assume that one reason for the discrepancy between the polls and the election result was that white supporters of Deukmejian were reluctant to appear intolerant to survey researchers and thus falsely reported being undecided. Other polls have similarly underpredicted the performance of white candidates running against African-Americans in other high-profile races, including the 1989 contest for Virginia governor and the '89 and '93 New York City mayoral elections.

But recent research has suggested that the "Bradley effect" has subsided. A paper by Harvard University's Daniel Hopkins analyzing state-level elections since 1989 estimates that the size of the Bradley effect -- which was never very big to begin with -- is now effectively zero. Similarly, no effect is found for female candidates. And despite much hand-wringing in the media of a Bradley effect hurting Barack Obama, he actually outperformed preelection polls on average by about three percentage points during the 2008 presidential primaries.

My analysis indicates that it is unlikely that recent polling on Prop. 8 in California substantially understates support for the initiative. Given how much the marriage ban is currently trailing in the polls, the probability is very low that a Bradley effect would lead to a Prop. 8 victory if the election were held today. Stay tuned, however: Both sides have raised millions to finance what will undoubtedly be a knock-down, drag-out fight via TV ads. A lot can change in seven weeks -- including polls. But when new surveys are released, take voters at their word: They are telling the truth.