Facebook A/B Testing – Complete Guide for New Campaign Improvements [2022]

Facebook ad testing is a must-do for all advertisers. Consider the following scenario…

You’ve launched your first Facebook ad campaign with great results so far.

Then, suddenly, all your key metrics begin to drop.

What’s happening?

The culprit could be a saturated audience–when your ads cycle through one audience with a campaign setup that isn’t reaching new people. Or the source of your declining results could be a Facebook algorithm change.

Often, the reason for negative shifts isn’t diagnosable at all.

The only way to keep ahead of campaign changes and consistently improve Facebook ad results is frequent A/B testing.

In this guide, you’ll get a solid foundation of Facebook ad testing knowledge, plus a few advanced-level hacks we love.

Or skip the step-by-step and jump to any section of this guide:

1. What is Facebook ad testing?
2. What can you A/B test on Facebook?
3. How to run an A/B test in Facebook Ads Manager?
4. What are the Facebook ad testing best practices?
5. TOP 5 Facebook ad A/B testing ideas

What is Facebook ad testing?

At the highest level, Facebook ad testing is an organized way to find better-performing campaign setup methods and ad creatives that appeal to specific audiences.

Technically, Facebook ad tests are structured A/B or split tests of two variations on the campaign, ad set, or ad creative level.

Over time, many successful A/B tests lower your Facebook ads cost significantly. For example, if your current CPA is around $10, it would decrease to $8.1 after five 5% improvements.

Not all ad tests end with a winning new version, and not all discoveries improve the conversion rate by 5% or more. That’s why you need to test frequently. You’ve got to take good care of your Facebook ad campaigns to get top results.

Below is the general workflow of a Facebook ad test setup. We’ll learn the details of each of them in the 3rd section of this guide: How to run an A/B test in Facebook Ads Manager.

1. Plan what you want to test.
2. Set up a new testing campaign in the Facebook Ads Manager.
3. Wait for the results to come in.
4. Measure and conclude.
5. Apply your new learnings to the existing campaigns.

While not rocket science, we get that split testing can be overwhelming for beginners. There are many best practices to keep in mind. There are also many different options to choose from.

What can you A/B test on Facebook?

You can test many facets of your Facebook ad campaigns on the campaign, ad set, and ad levels. Let’s take a look at the options within each.

⭐ Campaign level

Campaign objective: decides what kind of delivery optimization, placements, and budgeting options your ads can have. For example, you should select the Traffic option when you want to bring more people to your website.

Traffic is one of the top campaign elements that we recommend A/B testing.

Campaign budget optimization: determines how Facebook will allocate budget between ad sets in a campaign–automatically based on performance, or using manual ad-set-level budgets set by you.

⭐ Ad set level

Ad testing often makes the biggest impacts on the ad set level. Specifically, we see big improvements for Linear clients when we test new target audiences.

For now, a brief walkthrough of your ad-set-level options:

Bidding methods: select between daily and lifetime budgets and add a specific schedule for Facebook to show your ads.

Audience: choose who (and how many) you target, from Saved to Lookalike audiences.

Target audiences are another of the top 5 campaign elements that we suggest that you A/B test.

Placement: choose between automatic placements or limit your ad delivery to specific platforms and placements.

Note: generally, using automatic placements works best. We don’t advise using a lot of resources testing placements manually.

Ad delivery optimization: delivery optimization lets Facebook algorithms know what goal you’d like to achieve with your campaign. If you select “Link clicks,” Facebook will show your ads to people who often click on ads. However, if your goal is to get conversions such as Purchases, you should optimize for that event.

The delivery optimization events are well worth testing, and a great way to discover ad optimizations.

⭐ Ad level

On the ad level, ad creative is where a lot of your testing matters.

Getting your Facebook ad creatives right is crucial because it directly impacts how appealing (and click-able) your ads appear to your audience.

Testing new ad units is especially important when you scale a Facebook ad campaign.

As a general rule, testing regular ad variations is a great way to avoid ad fatigue (the plateau and decline of ad results that results from your audience seeing your ad enough to get bored with it.)

When managing our clients’ Facebook ad campaigns, test ad creatives is a key part of our campaign timeline.

With a moderate ad budget, you can aim to A/B test some new Facebook ad creatives once or twice per month.

Let’s look at the creative elements we can test on Facebook:

Ad format: lets advertisers test a bunch of ad types and formats. The most common ad format is a square, single-image ad. But recently, brands have added video and carousel ads into the mix.

Image or video style: style and color scheme decisions often feel subjective. Focus on meeting usability and brand standards to give your audience a cohesive and functional ad experience.

That said, don’t be afraid to test a few different design styles, from stock photos to single-color backgrounds and illustrations.

Headline, primary text, and link description: Facebook ads have three text sections that look different depending on the placement where your ad will appear.

Tip: The biggest game-changer is usually not the text surrounding your ad, but the text in your ad image. The ad image is what people notice first, so make sure you clearly communicate your offer in the image’s text.

CTA: the CTA (Call-to-action) button usually appears at the bottom of your Facebook ad. We don’t recommend testing it, as long as you follow genera CTAl best practices.

For example, select the most relevant CTA for your desired action. If you want people to sign up, use the “Sign Up” CTA, not “Learn More.”

Product sets for Carousel and Dynamic Product Ads: we’re entering the PRO-level territory. While this is relevant for e-commerce brands, it’s good to be aware that various product sets can be split-tested inside your Facebook product and carousel ad creatives.

Landing page: many marketers forget that the landing page is an integral part of their ad campaign. You can use Facebook ad A/B testing to discover new, top-performing landing pages.

Your head must be brimming with ideas about what to A/B test next. So let’s look at how to set up your test campaigns and some best practices that help you avoid rookie mistakes.

How to set up a Facebook A/B test?

You have the idea, the assets, and a hypothesis. You’ve even got the Facebook ad account and some extra budget to run some test campaigns.

So how to set up a Facebook ad test?

First, you should know that the best way to A/B test any Facebook ad campaign element is right in the Facebook Ads Manager. You won’t need any extra tools.

If you’re not aware of how the Facebook Ads Manager works, see this guide: Facebook Ads Manager: The Ultimate, Start to Finish Guide.

You have two options to begin a new A/B test:

• Test against an existing campaign
• Create a new test from scratch

Option 1: Test against an existing campaign

To test a new variation of an existing campaign, select the campaign you want to test against and click the A/B Test button.

Up next, you can choose what exactly you’d like to test.

Now, Facebook will ask you what you’d like to measure and how long you’d like your new A/B test to run. Once you’re happy with the setup, click “Duplicate campaign” and launch the test.

Tip: To avoid any bias that Facebook’s algorithms might have developed while learning about the previous campaign, we recommend creating the test from scratch. This leads us to the second option…

Option 2: Create a new Facebook split test manually

If you find Facebook’s guided split test setup confusing or limited, there’s another manual way to create a new ad campaign-level test.

Click the green “+ Create” button to prompt a popup window for campaign setup. Once you have chosen your campaign objective, you’ll enter the next stage of setup, where you can see the “A/B test” section. Toggle on the blue button.

Now, you can create the first campaign for your A/B test. After you complete its setup, Facebook will guide you through the second campaign’s creation.

Option 3: Test ad-set-level or ad-level ideas

The easiest way to do it is inside a single campaign. In this case, do not toggle on the “A/B test” feature when creating the campaign, but create several ad sets that will compete against each other.

Create a new campaign with two or more ad sets with different ad-set-level variables. If you want to test ad creatives, you can create a campaign with X ad sets that each include one different ad creative.

Here’s what your A/B testing campaign’s structure should look like:

Tip: Remember to turn off campaign-level budget optimization when creating a single ad campaign with various ad sets. Allocate an equal budget to each of your ad sets.

If you need additional guidance, check out Facebook’s brief A/B test setup guide.

While the setup process looks easy, questions will come up – “What budget should I use?” “How many variables should I test?” “What if there’s an audience overlap?” “When to conclude my test” – to name a few.

This next section on testing best practices should help.

10 Facebook ad testing best practices:

Let’s begin with a quick overview of the golden rules of A/B testing. We’ll dive deeper into each of them in a moment.

1. Test only one element at a time
2. Prioritize ideas with the highest impact
3. For non-tested elements, use your top-performing setup
4. Always create new campaigns for your A/B tests
5. Define your test’s metrics and how to define success
6. Set the right A/B testing budget for relevant results
7. Be patient, don’t conclude the test too early
8. Apply new findings to existing campaigns
9. Track and document your A/B tests
10. Always be testing, plan ahead

While you can occasionally omit a couple of these rules, making them part of your ad testing routine pays off.

1. Test only one element at a time

While it might be tempting to test all your ideas right away, it will be impossible to understand later what aspect of your campaign made the difference.

As the folks at CXL showed, scientifically testing multiple variables at once will lead to a complete mess.

If you want to test target audiences, ensure that the ad placements and creatives remain the same for both audiences.

If you’re testing multiple ad creatives, place each in a separate ad set, but make sure that the ad sets are identical across audience, placements, and budgets.

You get the point…

So how should you decide? Start by thinking about the potential long-term impact.

2. Prioritize ideas with the highest impact

Not all Facebook split tests are created equal. The potential positive impact on your advertising results (and budget) will vary.

If you’re not careful, you’ll spend valuable time making tweaks to a campaign without any impact at all.

For example, testing the CTA button would yield a much smaller difference than pitting two different ad creatives against each other.

The right-hand version of the Squarespace ad in the above image doesn’t exist – no experienced Facebook advertiser would test such minor tweaks.

However, Squarespace’s Facebook ad creatives do feature many different formats, colors, and messages.

Your Facebook split tests should aim to (eventually) gain at least a 10% uplift in results, not 1% or 2%.

When prioritizing your split test ideas, give extra points to those with the most compelling data to back it up, or greatest number of campaigns where the new findings would be applicable.

If you’re struggling to decide what to begin with, we recommend focusing on the target audience and ad creative split tests in the first place.

For more support on this step, hit up this guide to our preferred prioritization method, ICE scoring.

3. For non-tested elements, use your top-performing setup

Imagine that you want to A/B test a Lookalike audience v.s. a Saved audience. Or maybe you’re planning to try using specific ad placements v.s. automatic placements.

Whenever you test a single aspect of your Facebook campaigns, don’t just choose any random setup for the rest of the campaign. Keep it in the same ‘universe’ of your top-performing campaigns.

This rule is important because you’ll want to apply your winning version of the tested element to your existing ad campaigns.

4. Always create new campaigns for your A/B tests

Can you set up an A/B test inside an existing campaign that already has collected some data? The short (and correct) answer is “No.”

If you create an A/B test inside an existing campaign, you will get skewed results. That’s because Facebook’s algorithms have already learned how the past campaign setup performed and who was the best target audience for that particular setup.

Each A/B test should get an equal chance to all tested elements. That’s why they should all start from the same position – to have no previous learning data affecting the algorithms’ choices.

Again, set up new campaigns every time you want to run a new Facebook ads A/B test. 

Tip: Use the words “A/B test” in your campaign name so you can filter past test campaigns for future reviews.

5. Define your test’s metrics and how to define success

Before you begin a Facebook ad test, think about its ending. How will you know when to conclude the test? How will you determine which variant won the test?

Like all marketing and product management tests, your Facebook A/B tests should aim to bring statistically significant results.

This means that your test variants should differ by enough of a margin to be attributed to your change, rather than chance.

You can use a statistical significance calculator to understand how big of a difference in test results you need to consider the results decisive.

But which metrics should you look at?

90% of the time, your goal is to find ways to lower your acquisition cost, so avoid checking the vanity metrics like “Impressions” or “Reach” and zoom into down-the-funnel metrics like:

• Cost per landing page visit
• Cost per signup
• Cost per purchase

Occasionally, when testing ad creatives, upper-funnel metrics like CTR (click-through rate)  and CPC (cost-per-click) might also be relevant.

6. Set the right A/B testing budget for relevant results

Once you know the metrics you want to track and compare, you can calculate the expected cost of your split test.

A good rule is that the more variations you’re testing, the higher your required test budget will be.

Use the below formula to calculate the minimum test budget:

Minimum test budget = Cost-per-conversion x No. of Variations x No. of Conversions needed for statistical significance

You should collect around 50-100 conversions per variation. If the cost of a purchase conversion is too high, use another conversion event from the upper funnel, e.g., signups or leads.

Let’s say your goal is to get people clicking on your Facebook ad, and the average cost-per-signup for past campaigns has been $8.

If you’re testing 3 different target audiences, your test budget would be $8 x 3 x 100 = $2,400.

7. Be patient, don’t conclude the test too early

Even though you might be tempted to get your A/B test results fast and apply the findings to your existing campaigns, you should wait for at least 72 hours before drawing any conclusions.

It takes Facebook’s algorithms around 24 hours to start delivering a new ad campaign. It will take even longer for the same algorithms to learn about your creatives and audiences, and optimize respectively.

The more conversions Facebook algorithms receive, the faster they can exit the Learning Phase. It usually takes around 50 conversions to exit this phase.

Wait until all your test versions collect at least 100 conversions before you pause the campaigns. Afterward, it’s best to wait for another 24 hours, so that all conversions are tracked.

Tip: Plan 1 week for each Facebook ad test to have enough time for setup, tracking, and documentation.

8. Apply new findings to existing campaigns

You’ve run the A/B test, and you’ve got the results.

Remember that not all tests end with a clear winner and new top-performing findings. But what should you do when your test is successful?

If you have campaigns running for a long time, it’s best to apply the changes inside those campaigns. This way, you’ll benefit from keeping the past conversion data.

Do not make the mistake of launching new campaigns every week or month. Instead, use one primary campaign over a longer period and make changes inside that campaign.

If you launched your Facebook test on a new account with no other ad campaigns, you can simply pause the lower-performing variations and turn the top-performing campaign into your main campaign. Once again, you’ll keep the conversion data, which helps Facebook’s algorithms reach the right audience.

9. Track and document your A/B tests

Most marketers run A/B tests but forget to document them. After all, it requires extra time that you could spend on campaign optimization or other marketing tasks.

However, what if you change your PPC agency or the team member in charge of Facebook advertising leaves the company? You’ll be left with a big blank regarding what’s been historically tested. What a waste of good data!

This leads us to the following golden rule:

Always document your A/B tests.

You can create a simple spreadsheet to log your test ideas and results. Below is an example of reporting on an ad creatives test.

Not only will you keep all your learnings in one place, but you can also return to the spreadsheet whenever you’re looking for new ideas. You can see what worked (or didn’t) and focus on developing your testing hypotheses further.

Commonly used fields in a split test documentation sheet include:

• Test idea
• Hypotheses
• Setup description
• Planned dates
• Conclusion
• Key learnings
• Next steps

Note: there’s a free test tracker outline included in our ICE Scoring guide (linked in #2).

10. Always be testing, plan ahead

Setting up a Facebook ad test is only a small part of the process.

Before creating the test campaigns, you’ll have to get approval from other team members, confirm the budget, and possibly collaborate with the design team to receive ad creatives. This can take a week or longer.

To keep a constant flow of A/B tests running, plan each test at least 2 weeks ahead in time.

One of the easiest ways to keep all teams informed is to create a spreadsheet with test ideas, prioritization, and timelines.

🙌 Alright, now that you’re aware of all the Facebook ad testing best practices, let’s look at five test ideas that have the highest impact on your campaign performance.

TOP 5 Facebook ad A/B testing ideas

While you can test almost anything about your Facebook ad campaigns – from the audience to the CTA button or headline capitalization – there are some campaign elements that we’ve seen bringing the biggest change in long-term results.

For example, most people won’t read the text above your ad. But their eyes are set on the ad visual and the words written in the image or video. So running a Facebook ad test to define the best ad text is not as high of a priority as testing the ad creative itself.

AdEspresso examined the data from over $3M worth of Facebook ad tests and uncovered the campaign elements with the highest A/B testing ROI:

• Countries
• Mobile OS
• Age range
• Gender
• Ad image
• Ad headline
• Relationship status
• Landing page

However, some of the elements in the above list won’t be useful to you when it comes to A/B testing.

You already know your target audience’s location, age, and gender. So the real question would be how to reach the maximum number of your target audience members. Can you do it via a Saved or Lookalike audience targeting method?

Next, we’ll introduce you to the Facebook ad testing ideas with the highest potential gains.

Idea 1: A/B test audience targeting

The most important things to split test are your audiences and ad creatives. Let’s begin with the former.

When it comes to audience testing, we recommend that our clients test various audience types with little or no overlap. E.g., you could test a Saved audience v.s. a Lookalike audience or multiple different Lookalike audiences.

Make sure your tested audiences are more or less of the same size – don’t pit an audience of 5,000 people against one with 1M members. A good target audience size is anywhere between 500,000 – 5M people.

How to test multiple Facebook audiences?

To A/B test two or more Facebook audiences, create a new A/B testing campaign with several ad sets that each target a different audience. Make sure all ad sets have an equal budget and feature the same ad creatives.

For more inspiration and guidance, learn about Facebook target audiences here: Facebook Ad Targeting: Reach the Right Audience for More ROI.

Idea 2: A/B test your ad visual’s style

Your ad creative is the first thing people will notice when scrolling through their Facebook or Instagram feeds. Therefore, testing new creative ideas is one of the most efficient ways to improve your campaign’s click-through and conversion rates.

Also, there are so many things to test about the ad creative!

If you’re just getting started with Facebook ad testing, we recommend beginning by testing various visual styles. Let’s consider some A/B test ideas.

1. Test multiple color schemes to find what’s most attractive for your target audience.

For example, Canva’s running Facebook ads with mixed tones as different people react to different colors.

However, don’t go too far from your brand colors – you still want your Facebook ads to drive brand awareness and be recognizably yours.

2. Test your in-image text and messaging

Facebook ad creatives allow you to test various USPs (Unique Value Propositions). Many brands use Facebook testing as a method to quickly determine their most efficient marketing messages.

The best spot inside a Facebook ad for testing various messages is in the ad creative.

For example, Mailchimp’s Facebook ads feature a mix of value propositions that address marketers and salespeople. Notice that you’ll first read the in-image text and pay little attention to what’s written above and below the bright, colorful images.

Tip: When testing various messages, it’s best to keep all the other visual elements the same: vary the USPs but use the simliar size, color, illustration, etc.

3. Test illustration v.s. stock photo v.s. simple text-based visual

Different people react to different visual cues. Don’t get stuck in a single ad creative style. Try various ways to illustrate your key messages.

For example, Sephora advertises a mix of photos and illustrations to attract a broad audience with varied preferences.

Alternatively, you can test different models in your photos or various product illustrations. Look for the ones that bring results at the lowest cost.

Idea 3: A/B test your ad visual’s type & format

In 2022, video creatives have become a necessity for Facebook advertising. While you can still upload and advertise static ad creatives, there’s a high chance that well-designed, engaging video ads would yield a higher number of results.

An experiment by Biteable resulted in 3x more leads and 480% more clicks for video ads v.s. static image ads.

For example, Starbucks is advertising its rewards program with a mix of static and video ads.

When testing static creatives v.s. videos, try to keep the colors and messages of both test versions as similar as possible. Otherwise, it might be some other ad element affecting the conversion rates.

Once you’ve got a set of new ad creatives ready, create one Facebook A/B testing campaign and place each ad creative in a separate ad set. To keep the test budget on the low side, limit your test variations to 5-6 different creatives.

Idea 4: A/B test your target conversion event

When setting up your ad campaign, you can select between several delivery optimization methods.

You can optimize your ad set’s delivery on link clicks, landing page views, signups, purchases, or some other event.

While the best practice is to optimize your Facebook ad campaigns on conversion events such as signup or purchase, you might get interesting results when targeting other upper-funnel events.

This type of Facebook A/B test is especially relevant for brands that struggle to collect at least 50 lower funnel conversions per week. On such occasions, Facebook’s algorithms will struggle to exit the Learning Phase.

In this case, set up a couple ad sets with different delivery optimization events (e.g., landing page views, signups, and purchases).

Idea 5: A/B test your landing page

Your Facebook ad campaign doesn’t end the moment someone clicks on your ad. Your audience also needs to convert. A conversion (or desired action, like buying or signing up) happens on your landing page.

While you can use tools like Unbounce to set up a new landing page, working with an in-house development team or an agency to set up a new landing page version will be easiest.

For example, you could test your landing page:

• Headline
• Hero image (photo v.s. illustration)
• Signup form
• Section order
• Layout
• Length

How to A/B test your Facebook ads’ landing page?

You’ve got two options:

1. Use a landing page split testing tool like Unbounce or Google Optimize that will automatically send a set % of your page visitors to each landing page version.

2. Set up two Facebook ad sets with the same ad creatives, but with URLs linking to various landing pages.

Once you get into testing landing pages, you may benefit from some CRO (conversion rate optimization) help. If that’s the case, we’ve got you covered.

Time to get started with A/B testing!

You’ve learned all the Facebook ad testing best practices and top-performing split test ideas. Now, it’s time to roll up your sleeves and find new ways to optimize your ad campaigns.

If you need help launching your Facebook ad testing strategy, request a free growth proposal– a Facebook advertising expert is ready to help.