Platform ROAS answers a question you did not ask. It tells you how many sales the platform took credit for. The question that decides whether to keep spending is different: how many of those sales would have happened anyway, with no ad at all?
That gap is huge. When brands run real experiments, the measured incremental return comes in 30% to 60% below the platform-reported number, because Meta and Google routinely over-attribute by 20% to 60%. We explained why that over-attribution happens in why your ROAS is lying to you. This playbook is how you prove your own number with an experiment instead of trusting the platform’s.
01 What is incrementality testing?
Incrementality testing is a controlled experiment that compares a group exposed to your ads against a holdout group that was not, then measures the difference in conversions. That difference is the lift: the sales your advertising caused, stripped of the ones that would have come from organic, direct, email, or brand demand regardless.
It is the closest thing performance marketing has to a clinical trial. You are not modelling or attributing. You are withholding ads from part of the market and watching what does not happen.
02 The number that scares the platforms: iROAS
Incremental ROAS, or iROAS, is return calculated only on the sales the test proves your ads caused. Reported ROAS counts every credited sale; iROAS counts the incremental ones. Across tested accounts, iROAS typically lands 30% to 60% under the platform figure. A campaign showing 4x can be doing 1.6x in reality, which changes whether you scale it or kill it.
This is why incrementality is the causal-proof layer of the measurement stack. MER tells you the blended account is efficient or not. Incrementality tells you which specific channel earned the credit, with evidence a platform cannot argue with.
03 Two test designs, and when to use each
| Geo holdout | User-level holdout (conversion lift) | |
|---|---|---|
| How it works | run ads in some regions, suppress them in others, compare | randomly hold a share of the audience out of seeing ads |
| Best for | upper funnel: TV, YouTube, display, broad social | lower funnel: paid social, where the platform can split the audience |
| Tooling | your own region split, or a geo test tool | Meta Conversion Lift, Google lift tools, ghost ads |
| Main cost | lost revenue in the off regions during the test | smaller, the holdout is a slice of one audience |
| Precision | noisier, needs many markets | tighter, randomised at the user level |
Match the design to the channel. A geo holdout is the honest way to test channels that user-level tools cannot cleanly split. Conversion lift is the precise way to test lower-funnel social.
04 The playbook
1. Pick the question and the channel
Test one channel at a time, and pick the one where being wrong by 30% to 60% costs the most money. Usually that is your biggest single line of spend. Decide the design from the channel: upper funnel goes to a geo holdout, lower-funnel social goes to conversion lift.
2. Choose the holdout size
Hold out 10% to 20%. Smaller than that and the signal is too weak to read; much larger and you are burning more revenue than the answer is worth. For a geo test, that means choosing a matched set of control regions that look like your treatment regions on baseline sales and seasonality.
3. Size it before you launch
Confirm you have enough conversions or enough matched markets to detect the lift you expect to find. A test that cannot reach significance is just lost revenue with a number at the end. Plan for 4 to 8 weeks. Geo tests need the longer end to smooth out local noise and seasonal swings.
4. Run it clean
Change nothing else while the test runs. No new creative push, no promo, no budget swings in the tested channel. The whole value of the experiment is that the holdout is the only difference between the two groups. Contaminate it and you have an anecdote, not a result.
5. Read lift and iROAS
At the end, compare conversions in the exposed group against the holdout. As a rough bar, a 15% to 30% lift signals meaningful incrementality for a performance channel; brand campaigns often show 5% to 15% on sales and more on survey metrics. Convert the proven lift into iROAS so you have a return number you can trust.
6. Act on the truth, then feed it forward
Reset your target for that channel to its real iROAS, not its reported ROAS. Then feed the result into your marketing mix model as a calibration point, so the model’s view of that channel matches reality. A lift test you file away and forget changed nothing.
05 The cheap version every brand can run
Full geo holdouts and conversion lift tests have a size threshold. Under roughly $5M in revenue, the lost revenue from a clean holdout often costs more than the answer is worth, and you may not have the conversion volume to reach significance.
You can still run the poor-operator’s incrementality test: the blackout. Turn one channel off completely for two to four weeks and watch total orders and blended MER, not the platform dashboard. If you pause Meta and total orders barely move, that channel was claiming far more than it created. It is cruder than a randomised test and seasonality can muddy it, so run it in a stable period and repeat before you bet the budget. As a first causal read, it is close to free and often eye-opening.
06 When not to bother yet
If you are early, under a few million in revenue, or running two channels with thin volume, skip the formal tests for now. Start with MER and one blackout test. Save full incrementality for when your channel mix gets expensive enough that being wrong by 30% to 60% is a number that hurts. The method is a scalpel, and a scalpel is wasted on a problem a blunt blended number already solved.
07 What good looks like
A brand running incrementality well tests its largest channel once or twice a year, holds 10% to 20% out for 4 to 8 weeks without touching anything else, converts the lift into an iROAS it trusts, and resets targets to that number. The platform dashboards stop being the source of truth and become one more input to sanity-check against a real experiment.
08 Where to start
Pick your single biggest line of ad spend. That is the channel most likely to be over-credited, and the one where the truth is worth the most. If you are large enough, design a clean holdout for it. If you are not, run a two-week blackout and watch your blended numbers.
If you want help designing a test that will reach significance, tell us your channel mix and spend and we will scope the smallest test that answers the question. If you paused your largest channel for two weeks, how far do you really think orders would fall?
Sources: Meta Conversion Lift and open-source GeoLift documentation; Triple Whale, Haus, Amsive, and Eightx 2026 incrementality guides (iROAS gap of 30% to 60%, 10% to 20% holdouts, 4 to 8 week durations, 15% to 30% meaningful lift).
Want the full playbook?
The 12 Growth Leaks burning your ad budget — the same internal doc we hand to new clients on day one. One short form, no spam.
Get the playbook →FAQ
What is incrementality testing in marketing?
Incrementality testing is a controlled experiment that withholds ads from a holdout group and compares its conversions against an exposed group. The difference is the lift: the sales your ads caused, rather than sales that would have happened anyway. It is the most reliable way to measure true ad impact because it isolates cause from credit.
How is iROAS different from ROAS?
ROAS counts every sale the platform attributes to an ad. iROAS, or incremental ROAS, counts only the sales a holdout test proves the ad caused. Because platforms over-attribute by 20% to 60%, measured iROAS usually lands 30% to 60% below reported ROAS, which often changes whether a channel is worth scaling.
What is a geo holdout test?
A geo holdout test runs your ads in some geographic regions while suppressing them in a matched set of control regions, then compares sales between the two. It is best for upper-funnel channels like TV, YouTube, and display that user-level tools cannot cleanly split. Plan for a 10% to 20% holdout over 4 to 8 weeks.
How long should an incrementality test run?
Plan for 4 to 8 weeks. Geo holdout tests need the longer end to smooth out local market noise and seasonality, while user-level conversion lift tests can sometimes read faster if you have high conversion volume. Shorter than 4 weeks rarely reaches statistical significance for a DTC brand.
Is incrementality testing worth it for a small DTC brand?
Below roughly $5M in revenue, a full holdout often costs more in lost sales than the answer is worth, and you may lack the volume to reach significance. Start with MER and a simple blackout test instead, and graduate to formal incrementality testing once your channel mix is expensive enough that a 30% to 60% error is costly.