On Wednesday last week, Yahoo News had a story on their website which claimed that: “A YouGov poll showed 44 percent of Britons cried when news broke that the monarch had passed.” In effect what Yahoo were claiming was that roughly 30 million people broke into tears on hearing the news. Does that sound credible?
I went to YouGov to see where this poll was, who they had sampled, and exactly what the question was. Yahoo, unlike Critical Mass, does not always provide links to the information they use.
The “survey” they cited was actually one of YouGov’s Daily Questions, and it is worth examining a little what it was actually telling us, if indeed it was telling us anything. I have previously contacted YouGov for details of their polling technique, but this is not information they are keen to share.
The question asked on the YouGov website was: “Have you cried or become teary/welled up at any point due to the death of the Queen?” Notice that this is rather different from “cried when news broke”. Notice also that this is really a number of questions rather awkwardly combined into one. Crying is surely different from becoming teary or welling up. It also begs the question whether “at any point” is rather non-specific at a period in time when public emotion is being manipulated very overtly by blanket coverage of the event.
Who were these people?
We know, because they tell us, that 3,282 adults replied. But, who are these people? And, more importantly, how were they chosen? Is this poll in any way representative of the views of the British public? Bear in mind that Yahoo were claiming 44% of “Britons” not 44% of people who happened to be on YouGov’s website last week.
In fact they do tell us a little about the people answering this question.
If you use the tabs on their website they break it down by religion, politics (meaning party voted for at the last election), social class, and age.
What is quite clear, even from this data, is that, if you asked that question only of those aged 18-24, you get a very different picture. What we don’t know however is how many of the 3,382 were in each of these groups. There seems little doubt that the figure for all is being disproportionately affected by the views of the older age groups (50+).
Do 90% of the public support nuclear power?
Were it not for the fact that we are being force-fed positive news about the royal family at present and that a major news outlet reported it as if were a fact, this poll would be no more important than a poll reported in the Daily Express a couple of weeks ago in which 90% of the 987 Express readers who filled in their online poll said that the UK should invest more in nuclear energy. Indeed, have we not all filled in polls on Twitter and become excited by the results if more than 100 people join us in doing the same?
Polling is important in politics for a number of reasons. Importantly it gives us a sense of how parties are faring or whether a particular policy has public support. They can take us outside our political bubbles and connect, in some way, with those we may not normally converse with.
What’s the right answer? What’s the right question?
The question on crying over the death of the Queen is an interesting example of how the question is almost inviting a certain answer. That it never received the answer was probably more to do with the country being split on what the correct response to a royal death should be. I just wonder what the answer would be if you asked simply: “When you heard about the death of the Queen, how did you react?” I think, though I have no way of supporting this, many people would say “sad”, a small few would probably say “burst into tears” and a sizeable minority would say “I didn’t”. In other words, part of the problem with all surveys is that they manipulate you into categories that you might not otherwise put yourself in. I rather suspect too that in the media onslaught after the Queen’s death many people would have sought out YouGov as a way of doing something, but again I have no particular evidence for that.
But all of this assumes something rather important. Opinion polls measure public opinion. But as Peter Hitchens has said: “Opinion polls are a device for influencing public opinion, not a device for measuring it.” What he is less clear on is quite how this is managed.
Hitchen’s point is that polls, particularly political polls, can be manipulated to create an illusion of certain individuals or policies as being unpopular. When Yahoo was making the case that republicans were losing ground they cited 44% who cried. This is because, if you say 56% seem to have felt no great loss at the passing, their story does not quite hang together.
We saw during Jeremy Corbyn’s tenure as Labour leader how the media manipulated events, including polling results, to undermine his chance of becoming Prime Minister. As Kitty S Jones reported after the first leaders’ debate, a snap poll, produced by YouGov and widely cited by broadcasters, announced that Johnson won based on a poll that was carried out an hour prior to the debate starting!
It is worth reminding ourselves that most people have only a passing interest in politics. Even fewer are prepared to fill in polls. Indeed, before online panels became the norm it would be normal to report the overall response rate.
I was a carefully recruited member of the YouGov Panel
YouGov states that: “Over the last eighteen years, YouGov has carefully recruited a panel of over 1 million British adults to take part in our surveys.” I am a member of YouGov’s carefully recruited panel of one million adults and I can say I had no sense when I signed up on their website that I was being carefully recruited. However, in fairness to YouGov, I should point that in 2016 the Pew Research Centre found that YouGov polls were some of the most accurate in the business.
I am certainly not arguing that we should ignore all polls, any more than I think we should ignore news because we do not agree with it, but that we should treat all polls as no more than one variable in a multi-variable universe. To make this simple, you would not believe in UFOs because one person claimed to have seen one, but if thousands of people said the same thing you might at least think something was happening. Polls are largely the same. The poll cited by Yahoo News did not show that 44% of Britons spontaneously burst into tears. What it showed was that 44% of people in a sample from a sampling frame of around one million, who happened to be on their website on a particular day, said that they felt sad, and that could include actual tears.
Beware of putting too much faith in a single poll and be sceptical of the way in which questions are asked to elicit certain responses. But they remain an important background indicator to the real politics that affect us all.
Socialist of many years. Former Labour member. Currently presenter of The Socialist Hour.