Rosie Dawson considers the role of pollsters in perpetuating misleading narratives about religion and religious people.
The newspapers on 9th June, as you might expect, were full of articles analysing the fortunes of the political parties and their individual candidates. It wasn’t long, however, before I came across an assessment of the success of another group of people at the heart of the election, for whom the result really mattered – the pollsters, I read, had experienced “a mixed night.”
After wrongly calling the results of the 2015 general election, the EU referendum and the 2016 US presidential election, we could be forgiven for wondering whether the people who were meant to know what everyone was thinking knew very much at all. What had gone wrong? Was it the sampling methods used, or the wrong questions asked of the wrong people? Did people suddenly start responding in new and contrary ways to the questions asked?
Why this matters for pollsters was explained in a BBC Radio 4 programme by James Morris, then Labour’s polling adviser. Political polls are the publicity engine for market research companies; predict the election result correctly and you will be trusted by commercial companies wanting information on what products they can sell to whom. “It’s a way of getting your name in the newspapers, and the result of that is to turn your market research firm into a household name.”
Journalists need their news. We are now officially into the silly season; a few dramatic survey results between now and the start of the party conference season would cheer up the producers and copywriters who have to scrabble even harder than usual to fill the allotted newspace and airtime.
Polls have also been telling us about religion since Gallup in 1939 produced information about Americans’ voting intentions based on whether they were Catholic or Protestant. In 1944 it conducted the first poll asking Americans if they believed in God. 96% did. Polls about religion have been growing exponentially ever since, particularly with the arrival of internet polling. Are they telling us anything useful? What can’t they tell us? Which pollsters should we trust?
This year the market research consultancy ComRes has established a faith unit. It maintains that policy makers need to be better informed about the part religion plays in people’s lives, and that the public also wants to be asked more questions about it. A specialist unit will know the questions to ask to get the most precise results. So a poll ComRes conducted at Easter asking whether or not people believed in the Resurrection of Jesus differentiated between those who believed in the Biblical version and those who believed in the Resurrection but not as told in the Bible. Which is an interesting approach, although for awkward pedants like me still begs a lot of questions about the assumptions made about what the Biblical version(s) say and how they are being read.
It’s tempting to seize hold of the latest poll headline without reading down to the more nuanced questions some pollsters are asking; easy to skate over their different methods, or the aims of those who commission them. It’s not unkind to suggest that last month’s survey from Premier Christian Communications asking whether people felt Christianity is being marginalised, and which was conducted alongside a campaign for a stronger Christian voice in society, delivered the very result its creators wanted. This isn’t to dismiss the views expressed by a large number of people (12,000 and still counting) but I’m unconvinced by the argument that they are representative of anyone beyond Premier Radio listeners and their circles. Surveys which purport to reflect who we are as individuals and as groups, and how we relate to others, need to be meticulously reported, because polls are so often received as fact. They create reality and feed into our views of ourselves.
This is the point made by the eminent social scientist Robert Wuthnow in his book Inventing American Religion; Polls, Surveys and the Tenuous Quest for a Nation’s Faith. He argues that polls have led to America understanding itself as a nation of Evangelicals. In launching his campaign for the nomination for presidency in 1976 Jimmy Carter famously identified himself as a born again evangelical. Polling organisations rushed to find out more about this group and in so doing, according to Wuthnow, defined and over-exaggerated its importance, and this has had negative consequences for those who don’t fit into the narrative of how America understands itself.
Not everyone accepts Wuthnow’s argument but he’s surely right to suggest that people’s view of religion will be based in part on the opinion polls they come across, and that false opinion polls will lead to false narratives about religion being spread abroad.
Wuthnow’s critique is largely directed against quantitative surveys, but robust academic quantitative research into religious affiliation, worship and belief is indispensable in enabling scholars to map social change over decades. However rigorous, though, it can’t tell us everything about the everyday realities about what is now routinely called ‘lived experience.’ As the saying attributed to Einstein, and much beloved of qualitative researchers goes: “Not everything that counts can be counted and not everything that can be counted counts.”
More blogs on religion and public life…
From Genes to Values: Shifting the Designer Baby Debate
Simon Reader
Does Faith Make you Healthier and Happy?
Greg Smith
Dog Collars, Tower Blocks and Nation-building
Chris Baker
Not the Sun Wot Won it: Social Media and the Election
Rosie Dawson
Discuss this