Repairing Your Relationship With the Polls Post-2020

Few Americans made any conclusions on Election Night this year, having been forewarned by various experts, pundits, and candidates themselves that the election would likely take days to resolve. But one conclusion popped immediately: the polls were wrong.

As more votes are counted, that conclusion has held, although the size of the miss gets smaller as more ballots are added to the total. If you rely on data and message testing to inform your strategies—as we do—you are probably asking, “Can we even trust polling?”

In this post, we explore why it’s premature to be writing the eulogy on polling, and why advocates can (and should!) still rely on polls to inform their fight forward.

ELECTION POLLS ≠ ISSUE POLLS

A key difference between the polls that advocates use to strategize and the polls that are getting a bad rap in the wake of the 2020 results is, quite simply, the type of poll that it is!

Election polls that focus on the horse race are asking entirely different questions than the issue polls that ReThink and countless other organizations use in our messaging campaigns. Election polls ask you who you plan to vote for, and maybe some other questions, such as “how likely are you to vote?” or “is your vote in favor of your preferred candidate or against their opponent?”

Issue polls ask, simply, about issues: “do you support or oppose withdrawing all troops from Afghanistan by the end of the year?” or “which of the following do you think should be done regarding President Trump’s Muslim Ban?”

Can you spot the difference? Election polls ask people to indicate an action that they might take at a specific time in the future, whereas issue polls ask people to indicate thoughts and feelings that they have right now. With issue polls, respondents aren’t thinking about a high-stakes decision that is stress-inducing and potentially all-consuming. Rather, they’re engaged in a conversation on a topic where all they have to do is say what they think.

Most importantly, the accuracy of election polling relies on the respondent following through on the action they indicate they will take. This is why we say that polls are “snapshots in time”—campaigns will go to great lengths to influence your intended action, and the action you intend to take is not final until you take it.

WHO ARE YOU TALKING TO?

Election polls are (in)famous for the ongoing battle of the “likely voter universe.” When it comes to polling the horse race, everybody agrees that pollsters should be talking to likely voters—but nobody agrees on how to determine who a likely voter is.

Voter turnout is a bizarre thing. It can fluctuate unpredictably, and depend on a host of intertwining factors like who is on the ballot, what the major issues are, whether or not the election is regularly scheduled or a special election, how much money has been spent, and even the weather on Election Day. Some pollsters have meticulous formulas for determining likely voters, usually with some combination of previous voting history, donation history, volunteer history, registration date, and a whole host of other factors. Others choose to let the voters tell them: call as many people as you can, include the folks who say that they’re likely to vote, and toss the ones who say they’re not.

Needless to say, it’s tricky and often relies on a lot of assumptions. If you get the likely voter universe right, your poll will be widely celebrated for accuracy. If you get it wrong, you could be forever branded with the scarlet letter.

The most familiar example of this is 2016: most experts now agree that the primary reason polling models missed in that Trump v Clinton election is that pollsters didn’t include enough white voters without college degrees in their “likely voter” universes. In 2016, Trump’s campaign enticed many of these Americans to come to the polls, running against prior voting history and thus confounding polls.

With issue polls, we don’t deal with this problem. Issue polls typically collect samples of all adults or all registered voters, two populations that are far more stable and have much more readily available demographic details to inform you on whom to interview. We don’t need to make assumptions or guess.

By removing the guesswork, we eliminate a major opportunity for error that is often the reason election polls get it wrong and provide our advocates with better, sturdier data.

IT’S THE MOVEMENT THAT MATTERS!

When ReThink conducts issue polls, we do it in a particular way: we ask about the issue, then we present some messaging. Then we ask about the issue again, to see if we change anybody’s mind. That way, we can see what our strongest messages are, who our strongest supporters are, who our most persuadable audience is, and who is our opposition.

The good thing about this setup is that, even if the topline result on the original ask about the issue is a bit off from the real number, we still get a good read on how opinion moves.

If we ask, “do you support or oppose overturning the Supreme Court’s decision in Citizens United vs. FEC?” and get a result that looks like 53–47% support, we might think that looks a little low on the support side, and we’d probably be right. But after testing our messages, we might see support increase to 60–40%. Even if our original baseline is a bit off, this still shows us that our messages are strong enough to move some people toward our side.

If we see a big increase in support among people under age 30, that’s an actionable insight.

If we see a sharp decrease in support among politically moderate men in the Midwest, that’s an actionable insight.

If we see one message do particularly well with married mothers who are registered Independents, that’s an actionable insight.

In election polling, if you get the topline wrong, you’ve gone bust. In issue polling, if you get the topline wrong, there’s still a treasure trove of good information that you can use to win.

As we collectively move on from the 2020 election, it’s important to keep in mind that, even if the polling industry misses on the election results, there’s still a colossus of good that they can do for advocates who are trying to secure wins and action on our priorities. ReThink Media will continue to provide thorough and substantive analysis of polling to inform advocates on messaging and strategy as we navigate the new political climate that we will be heading into in the new year.

(Cover photo by Markus Spiske on Unsplash)