Guest Post: How Global Zero Used Facebook Ads to Test Messages and Visuals for Its Newest Campaign

Talking to people is one thing. Talking to people effectively is another thing entirely. Digital channels provide an accessible platform to do the former, but succeeding at the latter is far from guaranteed. Of course, a primary goal of any self-respecting nonprofit’s digital team is to ably communicate their organization’s message to a mass audience. So how does a nonprofit go from talking to people to talking to people effectively? One word for you: testing. 

Global Zero, the international movement for the elimination of nuclear weapons, is gearing up to launch an exciting new campaign. Through grassroots activism and education on the local level, we’re hoping to change the way grasstops leaders on the local, state, national, and, ultimately, international stage view nuclear weapons. But a movement doesn’t pop up overnight - we need to help build it form the bottom up. This requires education and mobilization of supporters, two things that can only be achieved by, you guessed it, talking to people well.

With these two goals in mind, Global Zero’s digital team set out to use testing to determine what messaging our supporters would best respond to.

Test Set-up

To figure out how to best mobilize our audience, there were two things we sought to learn:

• How to describe the issue our campaign is taking on.
• What kind of visuals drew the audience into our campaign. 

 

To address the first point, we came up with seven different terms. Four of these described ‘the nuclear system.’

• Nuclear Dictatorship
• Nuclear Monarchy
• Nuclear Tyranny
• Nuclear Status Quo

The remaining three were variations on President Donald Trump. With President Trump’s nuclear saber rattling and seemingly itchy trigger finger, he has become a focal point for the world’s anxiety about nuclear weapons. We sought to figure out the best term to describe the U.S. President with.

• Trump
• Donald Trump
• President Trump

 

Looking to the second point, we selected three different images that represented different aspects of the campaign we were crafting. 

This image captures general fears about nuclear destruction (Image 1):



This one features Global Zero activists and aims to link our campaign to the Resistance movement (Image 2):

Finally, this image was picked to test how well people’s anxiety about President Trump would draw them into our cause (Image 3):



With these different elements chosen, we then needed to decide on a channel to test them on.

We could have gone with email, testing the textual elements in subject lines and body copy, and featuring the images in the email itself. However, this would limit our testing to folks already on our list. A large goal of our campaign is to change the national (and international) conversation around nuclear weapons, which will require reaching people outside of our existing networks. To address the need to communicate with new people, we decided on Facebook Ads as the best platform to test on. 

For the test, we created 21 different ads under a single ad set. This allowed us to test every combination of copy and imagery possible. The ads directed to our evergreen petition in which the signer commits to fight for eliminating nuclear weapons. While this piece of content isn’t particularly compelling, in this test we cared more about the response to the messaging on the frontend than the action on the backend. 

We decided to optimize the ads for traffic directed to the petition site, so we could see which of the ads most compelled people to convert to the website.

Here is an example of one of the ads created for the test:

Facebook Ads has an extremely robust targeting tool. As was mentioned previously, we sought to exclude existing supporters to see the appeal of our message to a brand new audience. Given this, we excluded our current Facebook followers and their friends from targeting using Custom Audiences. 

The people we did include were those that were progressive-minded and politically-interested. To reach this group, we targeted followers of progressive politicians and organizations and people that Facebook has determined to be prone to progressive political activism. (As a side note, Facebook’s ability to provide targeting for behavioral tendencies is extremely useful for both testing and paid advertising.) We were sure to keep the same audience targeting for each of the ads.

With the copy, imagery, and targeting in place, we rolled out the ad group for release. We set the ad to run over the course of three days; this longer run was chosen to mitigate any unforeseen factors of time that may somehow bias the results of the test. 

For this whole test, we had a budget of about $350 to spend.

Results

Looking to the results of the test, there were three metrics that we focused on: 

Average Reach - The number of people that saw the ad at least one. This metric shows both how many people stopped to look at our ad and illustrates how much the Facebook ad algorithm decided to show our ad to folks. 

Average Cost per Click - The cost per link click. This metric is illustrative of how many people saw an ad and chose to engage with it (the lower the cost per result, the higher the percentage of people that saw the ad who decided it was worth clicking on).

Average Relevancy Score - A score on a 1-10 scale assigned by Facebook that “estimates how well your audience is responding to your ad.” This is an important statistic as it takes into account how well an ad is being engaged with and resonating with the Facebook audience.

Here are the findings for the different copy elements (variously describing the ‘nuclear system’ and Donald Trump): 

Average Reach:

Nuclear Dictatorship - 1,556.33
Nuclear Monarchy - 926.67
Nuclear Tyranny - 704.67
Nuclear Status Quo - 575

Trump - 12,174
Donald Trump - 6,306.67
President Trump - 973.33

Average Clicks:

Nuclear Dictatorship - 21.67
Nuclear Monarchy - 6
Nuclear Tyranny - 4
Nuclear Status Quo - 2.67

Trump - 173.33
Donald Trump - 144.33
President Trump - 10.67

Average Relevancy Score

Nuclear Dictatorship - 8
Nuclear Monarchy - 4.67
Nuclear Tyranny - 4.67
Nuclear Status Quo - 5.33

Trump - 5.33
Donald Trump - 7.67
President Trump - 7

Average Cost per Click

Nuclear Dictatorship - $00.32
Nuclear Monarchy - $00.46
Nuclear Tyranny - $00.68
Nuclear Status Quo - $00.41

Trump - $00.79
Donald Trump - $00.42
President Trump - $00.45

“Nuclear Dictatorship” was the most successful descriptor of the nuclear system - it had the best numbers in every single metric category.

In the variations on Donald Trump category, things are a bit muddier. “Trump” had the largest Reach. This tells us there’s a healthy appetite for the most casual, straightforward variant. However, “Trump” also had the highest average cost per click, which means engagement happens at a lower rate than other variants. Additionally, “Trump” had the lowest Relevancy Rating, although this could be chalked up to the fact that “Donald Trump” and “President Trump” have two words (as opposed to one) for Facebook’s algorithm to use and it is presumed that this increases their relevancy in the algorithm.

Regardless, “Trump” underperformed in the categories that mark effectiveness of engagement. 

The lack of a clear answer to which of the Donald Trump variants is most effective in some ways reifies something that’s kind of intuitive: You can call Donald Trump whatever you want as long as it’s clear you’re talking about Donald Trump.

And here were the results of the different image variants: 

Average Reach

Image 1 - 7,884.43
Image 2 - 605.29
Image 3 - 1,460.29

Average Relevancy Score

Image 1 - 6.43
Image 2 - 3.14
Image 3 - 7.57

Average Cost per Result

Image 1 - $00.41
Image 2 - $00.65
Image 3 - $00.49

Image 1 was the clear winner of this test. It had the best numbers in every category other than Average Relevancy Score, which it had the second best score in. We think this is because an image of nuclear destruction clearly communicates what the stakes in our fight are. 

Image 3, which features Trump, while slightly less successful, was successful in getting eyeballs on itself, but wasn’t as good at ginning up actual engagement. This may be due to an oversaturation of Trump-related imagery, meaning Global Zero (or other brands) would have to come up with unique ways to present President Trump visually to break through the noise. 

The response to activist-y Image 2 was comparatively negative. This type of imagery needs strong contextualization and also is better deployed on folks that are already at least semi-active (as opposed to new, unactivated targets). 

 

***

 

With these results, Global Zero now has a base on which to build the rhetorical structure of its latest campaign. While we will be moving forward and using these results to inform our current messaging, it’s important to remember that testing should be an ongoing process. We’ll continue to utilize Facebook ads and email A/B testing to further refine our messaging and learn when and where to deploy variants of our rhetoric. 

It should also be said that while digital channels provide an unparalleled opportunity to glean insights, field work provides an important complementary channel to learn more about people’s response to organizational messaging. 

In short, nonprofits should be using all of the tools at their disposal to learn how best to communicate to their supporters on an ongoing basis. Nonprofits are always going to be talking to people, so why not be good at talking to them?

Brendan McGuire is the Digital Engagement Coordinator at Global Zero.