What is incrementality?
Incrementality is the marginal contribution, or the true impact, that your ads have on a specific outcome, most importantly sales and revenue. Incrementality testing answers the question: did a specific marketing action, like your Meta ads, cause the customer to purchase or would they have purchased regardless if they saw the ad?
Knowing the answer to that question is quite powerful because it allows you to allocate your budget to the channels and tactics that are actually driving net new purchases and steer clear of ones that are claiming credit but not actually contributing to growing your revenue and profit.
Why is incrementality testing so important with attribution?
Attribution is a part of everyone's measurement solution, but it does have inherent flaws including:
- Attribution doesn't measure causation, just correlation.
- Ad channels' self-attributed reporting, like Meta and Google, are going to claim credit for every purchase they can tie back to a touch. It's not hard to see how this can inflate the credit these channels get, especially with exclusion challenges and generous attribution windows.
- Collecting every touchpoint at a user level across all channels is very difficult. This is for a variety of reasons and continuously becoming more challenging.
Incrementality tests complement attribution as they measure what attribution can't: casuation. Incrementality tests are a way to audit your attribution and validate your measurement.
How to run incrementality tests
The most practical and statistically rigorous way to run incrementality tests is using Geo-based testing, a form of controlled testing. Geo-based tests are only feasible if your business has a large enough sample size to be able to create geo pairs. A rule of thumb is that a business needs 3,000 or more orders a month to be able to run geo incrementality tests. However, the exact threshold for testing will differ from brand to brand, so an analysis of your company's sales data is needed to understand your eligibility.
Testing Steps
- Step 1: Pick what channel or tactic you want to test. This is usually done at a channel level, like TikTok ads, or a subset level like Performance Max. It's best to start with your largest channels/tactics, especially those you think in-platform and/or your MTA solution isn't reflecting the true impact.
- Step 2: Divide individual geos into two comparable groups based on the historic sales of each geo. This is a complex analysis and a data science background is needed to perform. The control geo will account for 10-30% of the channel's population. For most businesses, this will represent about 1-3% of total sales.
- Step 3: Define additional test details such as length of test. At WorkMagic we generally structure our tests to run for 3-4 weeks
- Step 4: Build the test in the ad channel according to the test details determined in the steps above. The test will run for a predetermined time period.
To properly run ongoing incrementality tests, you need a robust team of data scientists and data engineers working together with your marketing team. While you can do this in-house, it's rarely cheaper or faster to do so. WorkMagic automates the whole process, empowering marketers to launch incrementality tests in minutes, removing the need for internal data science and data engineering resources.
Because incrementality tests are a snapshot in time, our learnings show that you should retest each channel every 3 months to have the most accurate measurement.
How to optimize based on incrementality test results
Next up, is optimizing based on the findings. Remember, incrementality tests are run one channel/tactic at a time at an aggregate level, so knowing how to apply the learnings to optimize your media is not as straightforward compared to using multi-touch attribution. To solve this at WorkMagic, we take the results of your incrementality tests and combine it with our data-driven attribution model, giving each brand a custom incrementality-adjusted attribution model. This model gets applied to all your dashboards across all levels of granularity, from business level to ad level, making optimizing for incrementality simple.
This methodology allows marketers to optimize for incrementality in real-time at every level of granularity, just like they would with a regular multi-touch attribution model.
Final Takeaways
Measuring for incrementality has never been more important, or easier to do. Brands that optimize for incrementality measure for causation, restore the signal strength of their data, and overcome the natural limitations of standard multi-touch attribution. For brands with sufficient volume, incrementality testing should be a foundational component to their measurement solution.
Want to read more about incrementality? Check out WorkMagic to learn how incrementality testing can be part of your measurement solution.
