Frustrated pollsters find Americans harder to figure out

We have all grown up with that picture of President Harry Truman on the day after his surprise 1948 re-election gleefully displaying the early edition of the Chicago Tribune and its headline, “Dewey Wins.” The Gallup organization that year stopped polling a week before the election, imagining that Dewey would maintain his six point lead. In fact, Dewey lost by 4.5 %, embarrassing the presumptuous pollsters almost as much as the presumptuous Republicans.
Pollsters supposedly have learned a lot in the subsequent five decades, but the inherent shallowness of public understanding of polls and the statistical noise that is entering the survey business through demographic changes and growing respondent refusal rates should give us serious pause. Misuse and overuse of polls can destroy the chances of lagging candidates to get media coverage and to raise money, and thereby help warp the democratic process. Unfortunately, 1996 saw more polling than ever before.

Polls probably did not change the outcome of the presidential race, but some of their mistakes nonetheless rival those of the classic 1948 fiasco and should elicit similar review. The worst errors belong to the CBS/New York Times poll, whose final survey the Friday before the election had Bill Clinton ahead of Bob Dole by a whopping 18 points. On election day the actual margin of victory was eight points, making the gap between the last poll and the final outcome 10 points, nearly the same as Gallup’s in 1948. CNN/Gallup, using quick turnaround tracking polls, had a huge Clinton lead until the last 24 hours, when they suddenly showed a Clinton drop to an 11 percent lead. Did they cover themselves? Did the race really tighten overnight?

“That’s nonsense,” says John Zogby of the Reuters/Zogby poll, the one polling organization that hit the election results on the head (Clinton 49%, Dole 41%, Perot 8%). Zogby, a maverick, is the landslide election winner of the polling contest of 1996. Not only did he call this year’s presidential race right, this David also beat the Goliaths in 1992.

What’s more, Zogby does not need to hide behind a supposed last second change of public opinion to validate his results. His surveys showed a tightening presidential race for some weeks, largely because his organization did not try to “push” people in the “undecided” camp into a premature choice. He also is unusual in seeing party identification as a “lead variable,” indicating ultimate preferences of voters, rather than merely a “trailer variable.”

“I’m comfortable in suggesting that 80% of this business is science and 20% is art,” Zogby says. Getting the right sample is crucial on the science end, and other national polls, he says, may fail to do sufficient screening of respondents or fail to double check enough of their responses. Moreover, some polling samples have an inherent bias. In the case of CBS/New York Times, it has a Democratic bias; that is, it exaggerates the percentage of the population that is Democratic. (Zogby is quick to mention his own past as a Democratic activist, indicating that his criticism is of analytical, not political, bias).

Zogby conducts focus groups and does individual follow-up interviews with some poll respondents to test questions and to seek out reasons not obvious in formal answers. Such techniques suggest he mixes his “art” with his “science” far too readily, according to some critics.

But one of the greatest political seers of this century, the late Samuel Lubell, decided after the 1948 debacle to balance polling with impressionistic door to door discussions with voters. This is a time-consuming, expensive technique that only a few enterprising news reporters seem to do these days, and Lubell, indeed, is all but forgotten. But his unusual approach–which also added art to science–enabled Lubell to see deeper into elections than did his contemporaries. For example, in 1952 and 1956, he kept finding avowed Stevenson voters who confessed a secret admiration for “Ike”, and consequently he was able to forecast accurately the landslide proportions of the Eisenhower victories of those years. Close interviewing also allowed Lubell in the 50’s to see far ahead, predicting in his book, “The Future of American Politics,” the political realignment to the right that didn’t really become evident for another decade.

These days America is even more complicated. Populations mix as never before, so you can’t make safe assumptions about voting behavior from race and ethnicity, or class and educational level, as in the past. Americans also seem more suspicious than before, more harried by telephone callers and pressured by time constraints; so they also less likely to accept a pollster’s attentions. The good pollsters, therefore, like Zogby (and at the state level, Stuart Elway of Washington State’s Elway polls), know the importance of humility–and of employing a certain amount of art with their science.

Bruce Chapman

Cofounder and Chairman of the Board of Discovery Institute
Bruce Chapman has had a long career in American politics and public policy at the city, state, national, and international levels. Elected to the Seattle City Council and as Washington State's Secretary of State, he also served in several leadership posts in the Reagan administration, including ambassador. In 1991, he founded the public policy think tank Discovery Institute, where he currently serves as Chairman of the Board and director of the Chapman Center on Citizen Leadership.