You will be able to design an ad test that gives a clear answer.
Farah ran two ads side by side for a weekend. One showed a single blouse on a hanger with the words breathable work blouses. The other showed a woman stepping off a crowded train with the words still fresh at your 9am meeting, and it also used a different photo style, a different button and a different landing page. By Sunday night the second ad had four sales and the first had one. She declared the second the winner and switched off the first.
Two weeks later the second ad was doing no better than the first ever had. She had no idea why it had won, or whether it really had. Five sales over a weekend could easily have gone the other way by chance, and with four things changed at once, she could not say which change mattered. The test cost her money and taught her nothing she could use.
Not all tests are worth running. On a small budget you can only run a few, so start with the ones that could make the biggest difference.
Usually that means the creative or the offer. A different hook, a different angle on the problem, a different kind of proof, or a different offer can change results a great deal, because it changes who stops and who acts. Lesson 4.2, Broad targeting and why platforms now push it, explained that creative now does much of the targeting, so a creative test is often also an audience test.
Small wording changes, such as a different button label or swapping one adjective, rarely make a difference you could detect on a small budget. Leave them until the big questions are answered. A useful order is offer first, then the main creative angle, then format, such as video against image, and only then the details.
If two versions differ in one way, any difference in results can be traced to that change. If they differ in four ways, you cannot tell which one caused it, or whether some changes helped while others hurt.
So build your two versions to be identical except for the thing you are testing. Same audience, same placements, same budget, same landing page, same time period. If you are testing the hook, keep the rest of the video the same and change only the opening. If you are testing the offer, keep the creative style the same and change only what is offered.
Farah's rerun did this. Both ads used the same photo style, the same button and the same page. One opened with breathable work blouses, the other with still fresh at your 9am meeting. Now a difference in results would tell her which message worked.
Running two ads in the same ad set is not a clean test. The platform quickly favours whichever one it predicts will do better and gives it most of the spend, often before there is enough data to be sure. That is useful for performance, but it means the two ads were never shown to comparable people in comparable amounts.
For a fair comparison, use the built-in experiment tools. Meta offers A/B testing that splits your audience into separate groups, so each person sees only one version, and reports which performed better along with an estimate of how confident it is in the result. Google offers experiments that split traffic between your original campaign and a changed version, for testing things like bid strategies, landing pages or ad text. TikTok has its own split testing as well. Look up the current names and options in each platform's help centre.
These tools take care of the fair split. They do not decide what to test or how long to run it, which is your job.
The most common way to get a false answer is to look at results every day and stop as soon as one version is ahead. Early leads come and go by chance, especially with small numbers. If you stop at the first lead, you will often crown the wrong winner.
So write three things before the test starts. The measure: usually cost per result against your target from lesson 8.1, Start from what a customer is worth, not clicks or likes. The minimum before judging: a spend or a time period, such as the full test budget or a set number of weeks, that you will reach before deciding anything, unless one version is clearly broken. And the decision rule: what result will make you keep version B, keep version A, or call it a draw and test something else.
When the experiment tool reports a confidence figure, read the platform's explanation of what it means and treat a low-confidence result as no answer yet.
Farah's rerun used Meta's A/B test with an example S$300 budget split evenly over two weeks. Her measure was cost per sale, her rule was to keep whichever cost less if Meta reported reasonable confidence, and to treat anything else as a draw. Still fresh at your 9am meeting won, by less than the first test had suggested.
In the activity below you will write a test plan for two versions of one ad, with what changes, the measure, the budget and the decision rule.
Write a test plan for two versions of one ad, with what changes, the measure, the budget and the decision rule.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).