Why Polls Are So 'Scientific' and So Confusing
Problems: what you ask, how you ask it, voters who haven't made up their minds, and margins of error
THE conflicting poll results coming out of New Hampshire all week were enough to boggle any voter's mind.
In one poll, Republican presidential candidate Bob Dole was up by 10 points. In another, he was in a dead heat with Pat Buchanan. In yet a third, the Senate majority leader had slipped to second place. And all within a day.
''What we have ended up with,'' says pollster Peter Hart, head of Peter Hart Research, ''is a multitude of tracking polls that tell us irregular heartbeats with split-second regularity.''
Polls are designed to take a snapshot in time, to relay the pulse of the country on key issues or a candidate's standing. They have become indispensable tools for political strategists and an integral part of political reporting, driving the headlines and pointing the spotlight where the media's attention will go next. Critics charge that polls have pushed substance off the front pages and turned the nation's leaders into followers.
Polls' proliferation and their sometimes contradictory results have also led to a growing public distrust of them.
''People say, 'It depends on the question; you can get anything you want out of a poll,' so they tend to discount it,'' says Tom Kiley, a pollster with Martilla and Kiley, a Boston-based media-consulting firm. ''It's part of the larger antipolitical mood.''
Polling has, in fact, become much more exact since it burst onto America's political landscape in 1936. That year, George Gallup bragged that, using mathematical probability, he could predict the outcome of the national election by interviewing only 1,000 people. He forecast a landslide by incumbent Franklin Roosevelt. The pundits thought he was crazy, in part because an unscientific Literary Digest straw poll several months before had predicted a huge win by Republican Alf Landon.
Gallup proved them wrong.
''It was the object lesson of all object lessons about probability sampling,'' Mr. Kiley says. ''And Gallup has gone on to do polling ever since and typically is within a point of the election.''
Pollsters constantly work to find the most exact methods to gauge the public's pulse. They use computers to analyze census and other data to ensure their samples reflect a community's demographics. They use ''random digit dial'' to ensure they include people with unlisted phone numbers and those who recently moved. They try to be sure the wording of their questions is as neutral and precise as possible.
''Small differences in wording can make a difference,'' says Andrew Kohut, director of the Washington-based Pew Research Center for the People and the Press.
A striking example was this fall, immediately after President Clinton announced he was committing American troops to Bosnia. The ABC/Washington Post and CBS/New York Times polls both found the public dubious, but a Gallup poll found a much higher approval rating. The difference: The network polls' question included the fact that 20,000 American troops were going to Bosnia; Gallup's did not.
''In areas where public opinion is not crystallized, wording can mean a great deal,'' Mr. Kohut says. ''But in some areas where the public opinion is strong, no matter how you ask a question, you get roughly the same answer.''
This fall, House Speaker Newt Gingrich (R) accused the CBS/New York Times poll of asking an unfair question about how people feel about cutting Medicare in order to help balance the federal budget. The poll found it was an unpopular idea.
Mr. Gingrich charged that if the pollsters had asked about cutting the ''rate of increase'' in Medicare, the outcome would have been very different.
''In point of fact, our polls used exactly the wording that Gingrich had suggested,'' Kohut says, ''and we found exactly what The New York Times found.''
Unethical or inexperienced pollsters can sway the results of a poll, which is why professional pollsters warn consumers to look skeptically at the source of any poll and the kinds of questions asked.
That's particularly true in New Hampshire where, with the exception of the few weeks leading up to the November election, there is more intense polling and attention to their results than at any other time during the presidential campaign.
All the major candidates have pollsters, as do all the major media outlets. Candidates usually hold their results close to their vests (unless their polls show a sudden favorable surge). But there are still enough press polls to feed the confusion.
Gallup, which works for CNN, is one of five firms that conducted tracking polls for the media in New Hampshire last week. (Tracking polls survey fewer people over a several-day period and report an average.)
Gallup had Dole way ahead of his challengers immediately after the Iowa caucuses. That was markedly different from three other tracking polls that had him in a dead heat with Buchanan.
''The truth is, polls have margins of error, and margins of error are not a joke: This is all mathematical,'' says Gerry Chervinsky, president of KRC Research, which polls for The Boston Globe and WBZ-TV. His polls supported Gallup's findings.
Mr. Chervinsky says the results could be due to the wording of the questions, the time the surveys were done, and the number of independent voters included. But he also points out that if you take a 5 percent margin of error into account, all the polls paint roughly the same picture.
''If you look at the [Boston Herald] numbers, where Dole is at 23 percent, that's within the margin of error where I have Dole at 27,'' he says. ''Then you have Buchanan at 22 - that's within the margin of error where I have him at 17.''
The conflicting results also tell a straightforward story about voter volatility and a lack of passion for the candidates who, pollsters say, have alienated the public with their negative advertising.
''There are no issues being talked about by these candidates, and polling can't create that,'' says R. Kelly Myers, director of the UNH Survey Center, which is polling for the Boston Herald and WCVB-TV.
Mr. Myers and others suggest the candidates and the media focus on a different set of poll results, such as the ones that show voters are fed up with negative ads and the relentless focus on the horse race.