Opinion: Opinion polls

POSTED: 02/8/12 3:44 PM

Opinion pollsters present non-results as news, Tom van der Meer, who teaches political science and methodology at the University of Amsterdam, wrote in an opinion piece in the Volkskrant yesterday – and the media lap up all this non-news. This is the text of his column.
“The electoral jump of the Socialist Party (in the opinion polls) was quickly followed by open doubts about the value of opinion polls in several newspapers and magazines. Their most important complaint: opinion polls are not real news.
The timing if this criticism is remarkable to say the least. Week after week media make much of the movements in the opinion polls. And often this is not correct. But now that the polls finally show a substantial (and significant) change in the electoral sentiment, those same polls are doubted. It seems like a case of it is not good or it is not valid. At least this sums up the reaction by Maurice de Hond on the criticism. But there is more going on: the media simply handle opinion polls badly.
Let me say first that opinion polls are useful because they characterize the sentiment among the population. We cannot do without them anymore. Maurice de Hond’s beautiful background analyses are often relevant, but they draw less attention than the random seat-numbers of the week. One could have endless justified reservations about opinion polls (to name just two: Sample surveys are never representative and many voters simply do not know whom to vote for), but they help journalists, scientists and policy makers to keep a finger on the pulse of society and to spot changes in the social opinion climate. That could be news.
But too often it is no news at all. At least, not in the bi-weekly polls, significant changes (like the one of the SP) are rare. In the majority of cases the polling bureaus report small movements of one or two seats. Those fluctuations are so small, that they are probably random. Often a chance ( because: random) poll with just a few different types of respondents is enough to win or lose a seat. It is obvious that in such cases we cannot assume that also something happened in the complete Dutch population. We are after all not interested in the 1,000 or 4,000 voters the polling bureaus queried, but in what the polls say about the complete population of eligible voters.
Often the differences are too small to say something meaningful about them. They fall within the so-called margins of error that are part of opinion polls. If a poll among 4,000 participants gives 31 seats to the SP, the conclusion cannot be more accurate than that the SP on a national level will score between 29 and 33 seats. When the SP scores the next week one seat more among 4,000 different respondents, it remains within the same margins of error. In that case we cannot say that something has changes; the chance that it is a random event is simply too big.
Still opinion pollsters, Maurice de Hond included, present such non-results as news. The media lap it all up with all the consequences.
A typical example of this we saw in the weeks before the 2010–elections. De Hond was allowed to present daily new polls in the TV-program The World Keeps Turning. While the daily changes were almost always too small to justify meaningful conclusions, this did not stop any of the participants in the TV-program to analyze endlessly why party A lost a seat, while party b won one. It could be due to a small blunder, a little scandal, or a sound bite. And all this while there were no indications for a substantial change in the Dutch population.
The damage is done by then. Politicians are interviewed about their so-called success in the polls and they start behaving accordingly. Just look at all the attention the SP received the past week, and the way other parties reacted. To belong to the winner, part of the electorate follows the party with a successful reputation (a phenomenon known among political scientists as the bandwagon-effect; currently in the American pre-elections it is known under the magic word momentum). In the end opinion pollster create this way part of the sentiment they attempt to report about. The trends in the polls become a self-fulfilling prophecy. And under those circumstances the media do not present news, they create it.
For this reason it is prohibited in Luxemburg to publish opinion polls within a month before elections. But that is not a solution.
It is the tragedy of the opinion pollster that there are doubts about poll-results, at a moment when there is finally a real newsworthy change among Dutch voters. This does not change the fact that the fundamental criticism is justified: weekly or bi-weekly opinion polls seldom offer news, simply because small changes are random.
The solution is simple: report the margins of error. This has been the custom in the United States for years. All changes that fall outside the margins of error deserve attention. This is first and foremost the task of the media. But while these problems have been pointed out to them many times in the past, they keep publishing juicy non-news. This is why the opinion pollsters also have to take their responsibility and enforce correct reporting. Otherwise De Hond remains the boy who cried wolf too often.”

Did you like this? Share it:
Opinion: Opinion polls by

Comments are closed.