Printer Friendly Page Can Polls Educate People? ~ #19

Can Polls Educate People? ~ #19

By Alan F. Kay, PhD
2003, (fair use with attribution and copy to authors)
Mar. 17, 2003

 Imagine this. You have a great idea for solving an important national problem. A few people have heard you and agree. Press, radio, and TV people tell you if you could get support from some top leaders, they would get your story out across the country. Leaders tell you they'd back you if you had a proposal with wide public support. But how can you get that support? Advertising and public relations gurus tell you a campaign to get your idea into the hearts and minds of the nation would cost $50 million. Nobody you contact has that kind of money. You're frustrated.

Ganymede, a highly qualified pollster, tells you that she can do a scientific survey for $50,000. She persuades you that with the survey findings you can raise the big money required to get the support of the whole country and of our leaders. You talk her down to $30,000 and she accepts.

You luck out. Ganymede does a smart, honest job and finds the best version of your proposal that a whopping 75% of Americans will favor strongly enough to tell politicians to vote for it. You attract a small following who do a great job of getting audiences of all kinds to understand and support your idea. There is soon no doubt in your mind that 75% of the people do support the proposal, even as opponents are speaking out against it. The media, whose stories until then were still-born, now can craft a story balanced by disclaimers and by quotes from you and your opponents, that will be seen as hard-headed. You get a few not-very-favorable stories.

You never get the promised support of political leaders. You are right to suspect that politicians are unreliable. But, it also may be that Ganymede overlooked one little known technique in survey research and you paid the price for her mistake.

Before any mistake, she did many things right. She tested support for your basic idea and variations thereof, until you agreed with her on the best formulation of the proposal — the most understandable, accurate, and highly supported version. Then, after presenting both pro and con arguments in a variety of ways, she evaluated how that affected support for the proposal. This method is called "the debate format." The findings from the debate format can tell you — for various audiences — both the best arguments to use to support the proposal and the best counters to use against your opponents' arguments. Armed with this information, you could predictably get 75% of an audience to favor your proposal.

What Ganymede did not take into account was that your opponents could similarly get any audience they addressed to have enough doubts about your proposal to bring support down to, say, 50%, and that fact could have been unearthed in the original survey if she had asked a few additional questions.

The evidence for this came originally from a series of ATI surveys on government reform. Using the debate format in 1993-1994, ATI tested four government reform proposals:

(1) Term Limits — would limit the number of years a person can serve in Congress. For example, after six or 12 years, members of Congress would have to leave office and not run again.

(2) In-District Contributions — would require candidates for Congress to raise, at least, one-half of their campaign funds from individual voters in their districts.

(3) Congressional Office of Public Opinion Research — would require Congress to fund an independent office, set up to conduct scientific, non-partisan, large sample, surveys of public opinion on all important national issues and to promptly release the results to the media so that Congress and the public will know what most Americans want for legislation.

(4) In the same way we've developed and use the Gross National Product to measure the growth of the economy, National Quality-of-Life Indicators would develop and use a scorecard of new indicators for holding politicians responsible for progress toward other national goals, like improving education, extending health care, preserving the environment, and making the military meet today's needs.

The complete findings for each of these proposals, and almost 50 other government reform proposals, covered in this series, would be far too lengthy to include here. What we can do is show how hearing the extensive pro and con arguments moved various percentages of the people to increase their support, decrease it, or not change at all.

(1) (2) (3) (4)
Increase 19% 36% 17% 21%
Decrease 30% 23% 33% 31%
No change 51% 40% 50% 48%
Net change +3% +9% -5% -2%

For example, column labeled (1), Term Limits, 19% of respondents increased their support; 30% decreased their support; and 51% made no change. But, the net change was from 71% to 74%, or +3%. If you knew only the net change of 3%, you might think that meant that very few people changed their minds on term limits. In fact, for every one who changed the net support, there were roughly 16.33 (49/3) altogether who chose to move. Those moving up were largely offset by those moving down. The comparable ratios for all four proposals are: (1) 49/3= 16.33, (2) 60/9=6.67, (3) 50/5=10, (4) 52/2=26.

[Note: A scale of 0-to-10 allowed respondents to increase and decrease support in varying degrees, a lot, only slightly, etc. In proposal (1), on balance, those decreasing, although more numerous, did not feel as strongly as those increasing. On balance, the net change was positive.]

The phenomenon illustrated by these findings is called "dynamic equilibrium" and is thought to apply not only to survey interviewers question-and-answer dialogues with the public but also to all audiences addressed by speakers. If the audience hears only one viewpoint, its collective opinions shift toward agreeing with the speaker, for example, the 75% approval that Ganymede promised you. The same audience upon hearing only the con side can drop down to, say, 50% support for the proposal. But going on at the same time, there are typically many other audiences around the country, some increasing their support, some decreasing it. The total system thus appears to be nearly in equilibrium. Collectively, people seem to have largely made up their minds and don't change much on average, but that view is erroneous.

Over a period of years, the public's support for a proposal or, for that matter, any idea may slowly drift up (or down) as the world changes. And if there is a major event that affects most people's thinking a lot, there can be larger jumps at any time. But even with regard to a proposal whose support level seems to be hardly changing, whenever the issue is in a state of "dynamic equilibrium", the public may be seething with changes that produce little net change. Failing to look into the net change of support for your proposal, the great pollster, Ganymede, made a key mistake.

The dynamic equilibrium phenomenon also teaches that people cannot really be educated by poll findings or by speakers with differing points of view. Speakers can cause a lot of changing to and fro of minds, but true education, whether for adults or children, is part of a complex, life-long process.



>>> 2.5  The Polling Critic

Table of Contents

Return to Top