Printer Friendly Page Telling a Good Poll from a Bad Poll ~ #1

Telling a Good Poll from a Bad Poll ~ #1

By Alan F. Kay, PhD
2002, (fair use with attribution and copy to authors)
May 21, 2002

Most heads-of-state claim that majorities of their people approve of their policies and actions. Even dictators seek to legitimate their power based on similar claims. Opponents also claim that the people are with them. Who is right, leaders in power or opposition? Often, it turns out, neither. Large majorities of the general population worldwide have views different from leaders both in and out of power. This leads to enormous disconnects between the leaders and the led that are not alleviated as regime follows regime — even in democracies.

For example, polls conducted by a Canadian firm, Environics, in twenty countries around the world over the last five years have shown repeatedly that people everywhere want better education, health care and environmental protection. Yet governments spend more on the military and large projects of dubious value to ordinary people but financially valuable to elites and special interests.

There is only one practical way to find out what people want for governance: careful, high quality, scientific, random sample polling. Today in 60 countries there are pollsters who are capable of conducting such polls with remarkable accuracy. Why are their findings not better known?

One reason is that many polls are never made public -- for example, those that leaders frequently commission to find out what to say to solidify their support. Secondly, a properly conducted, careful poll is labor-intensive and with 500 to 1500 interviews, too expensive for most purposes. Thirdly, polling is a competitive industry and commercial pollsters in all countries reflect the biases of their sponsors just to stay in business.

Furthermore, the mainstream news media, press, TV and radio, reflect the biases of their owners, advertisers, and political leaders in countries where they distribute or broadcast. This upstages findings from even the highest quality, most authoritative non-commercial polls that show the disconnect between leaders and public. This is especially true in the West, particularly the USA with its commercial "sound byte" news coverage.

The situation is not entirely hopeless. Sometimes commercial pollsters do high quality polls in the public interest. A few non-profit pollsters do so too -- unfortunately with generally limited access to mass media. Public-interest polling often comes up with remarkably different results than those of the commercial firms such as Roper, Harris, Yankelovich, and, best known around the world, Gallup.

For example, over a decade ago before the world knew of the Internet or heard of globalization, I and my colleagues conducted high quality surveys in the USA that first characterized globalization as these seven developments:

  1. Pollution crossing international borders;
  2. Global arms sales and arming of third world countries;
  3. Multinationals manufacturing in countries with cheap labor and weak environmental laws;
  4. Instant 24-hour trading of stocks, bonds, and currencies around the world;
  5. Designing, manufacturing, and marketing a global product in many countries;
  6. Workers all over the world going to other countries to work;
  7. Global news, advertising, entertainment, and information programs and software.
We found that the US public was well aware of these developments. When asked in random order about regulating them by international agreements, the top ranking: international pollution, produced a whopping 90% in favor of "strict or moderate" regulation. The remaining 10% were those who favored "mild regulation" or "not seeking new international agreements" and also included those who did not know or refused to answer. Asked the same way, the next five drew majority support for "strict or moderate" regulation: question 2 by 83%, 3 by 73%, 4 by 66%, 5 by 64%, 6 by 52%. Only support for the 7th, regulating globalized mass media, was a minority, 34%, not surprising in view of the devotion to "freedom of speech" in the US.

The polls showed that the well-educated were more favorable and familiar with these aspects of globalization and more aware of their opportunities. The less-well-educated knew back then that they would be more vulnerable. They were proved right by many studies in the following years. These have shown the uneven impacts of globalization on the poor, the uneducated, or those in regions bypassed by global financial and electronic networks.

Since then US opinion favoring regulation of globalization has been forced into mass movements, such as the many thousands whose leaders attended the Porto Alegre, Brasil, World Social Forums in 2001 and 2002.

In subsequent years, our further surveys showed a growing understanding of globalization by the US public and led to uncovering support for a pragmatic approach to address the bad news of globalization and unfettered world trade. One poll question was formulated this way:

"Trade agreements are crafted by economists who focus on economic aspects. The economists get very little input from other scientific advisors, like anthropologists, social scientists, and ecologists who often do see ways to protect a country's social institutions, culture, economy, and environment. Do you think that experts in other social and physical sciences should be involved in the development of trade agreements, or should the agreements be designed by economists alone, and not be complicated by competing viewpoints?"

A huge 71% responded that other experts should be involved, while only 23% said economists should design agreements alone.

We then asked about support of two different viewpoints in random order.

A solid 65% agreed with this NGO view: "Some people say that a combination of economists and experts from social and physical sciences would produce trade agreements more acceptable to everyone. The amount of time spent as these professionals learn to work together would be wisely invested, as they are sure to be more successful than the current system of economists working alone. Reaching agreements using only the limited ideas of economists will mean continuing delays in treaty approval, as well as harmful social and environmental impacts."

Only 26% agreed with this elitist view: "Other people say introducing non-economic considerations will make these already complicated negotiations hopelessly more complicated, so that no agreements will be reached for an even longer time than it would take for the economists to put together satisfactory agreements. With every year that it takes to reach satisfactory agreements, each country's economy will suffer from the lost opportunities for expanded trade, jobs, and a better material standard of living."

The WTO, which only emerged from GATT in 1995, was built on the elitists' narrow views. It took the NGO protests of Seattle, London, Prague and Genoa to force "free trade" ideologues to pay attention to human rights, labor, and environmental issues. In our "debate" format poll, support for the original question on broadening trade agreements remained unshaken. This resistance to counter-arguments and persistence over time is characteristic of public-interest polling.

To learn more about public-interest polling around the world, stay tuned to these columns.

>>> 2.5  The Polling Critic

Table of Contents

Return to Top