Understanding incrementality: The key to measuring a campaign’s true impact

To help advertisers pinpoint the exact impact of their campaigns, this guide provides an overview of incrementality testing methods.

What is incrementality? 

Definition  

Incrementality assesses the true impact of an advertising campaign by isolating the outcomes definitively resulting from the marketing activity from those that would have happened without the campaign. In other words, it answers the question, “Would I have gotten those sales anyway, without running ads?” This method helps advertisers understand how much of their success can be directly attributed to a specific campaign.   

“True incrementality testing requires a fully isolated holdout group with zero impressions. Without this, results are biased and can’t accurately measure incremental impact. Advertising-free holdouts ensure any uplift is due to the campaign, not external factors. This method definitively proves incremental value.” Martin Froschauer, Senior Product Manager, Measurement 

The importance 

Understanding incrementality is crucial for advertisers that want to understand the true value of their marketing efforts. It provides clear insights into return on investment, helping to evaluate the impact of specific channels and partners, and enables brands to prove their campaigns’ effectiveness and make informed, long-term budget decisions. 

How to measure incrementality 

Measuring incrementality involves designing a test that can isolate and evaluate a specific campaign’s impact on a desired outcome, such as sales or user engagement. Start with randomly assigning participants to a treatment group that receives ads and a control group that doesn’t. Then, compare the outcomes of the two groups before, during, and after the ads are delivered to determine incremental effect. 

For larger clients, incrementality tests are particularly effective due to the higher data volume and complexity of their marketing mix, which often includes multiple channels.

How does incrementality differ from traditional attribution methodologies? 

Marketers should consider both attribution and incrementality when measuring campaign effectiveness. Attribution is useful for short-term optimization, allowing for quick adjustments based on daily or weekly results. Incrementality, in contrast, provides long-term insights, typically reviewed annually to assess overall channel performance. Combining both methodologies offers a more comprehensive view than relying on just one. 

Common attribution models: 

  • Post-click attribution: assigns all credit to the channel a user interacted with after a click, focusing on actions taken until conversion 
  • Last-click attribution: assigns all credit to the final click before conversion often  overvaluing the last touchpoint and undervaluing earlier interactions  
  • Multi-touch attribution: spreads credit across all touchpoints, but may not fully capture the true incremental impact of each touchpoint.  

While attribution focuses on assigning credit to specific touchpoints, it doesn’t confirm the added value of marketing efforts. Incrementality, however, measures the difference between a treatment group exposed to the ad and a control group not exposed, offering insights into net overall lift effectiveness.  

Advantages of incrementality:  

  • True impact: isolates the effect of an ad to determine outcomes directly attributable to a marketing effort. 
  • Causal measurement: Reveals how much of an observed outcome can be directly linked to the campaign, compared to just showing correlations. 
  • Clear ROI calculation: offers a clearer understanding of return on investment by showing the actual contribution of each marketing activity. 
  • Budget optimization: Focuses on driving new conversions that wouldn’t have happened without the ad, preventing wasteful spending.  
  • Better decision-making: Provides precise data on what drives results, enabling better budget allocation. 
  • Enhanced campaign effectiveness: enables advertisers to optimize strategies by understanding the true impact of different channels and tactics.  

Understanding various methodologies for incrementality testing 

There are several methodologies for incrementality testing, varying in complexity and suitability for different environments.  

Geo-based experiments segment users based on geographic regions to form control and treatment groups. This approach is privacy friendly as it does not rely on individual tracking.  

The most common geo-based experiment is Geo-Lift Testing.

User-level experiments involve randomizing the audience at an individual level to create test and control groups. These experiments often use pseudonymized user data to protect privacy while ensuring precise measurement of ad impact on specific individuals. However, privacy regulations must be carefully adhered. 

User-level experiments include: 

                    Intent to TreatPublic Service AnnouncementGhost AdsGhost Bids
What is itBasic test that splits the full audience into control and treatment groups Serves real ads to both treatment and control groups, with control group receiving non-branded adsWhen a control group user would have been shown an ad, a “ghost” ad is recorded instead of a real adAdapted from Ghost Ads, tracking control group bids and recording “ghost” impressions that don’t result in actual ad placements, tracking potential exposure
When to use itAcross open and closed environments when no cross-site user IDs available Open environments with limited data volume; good for faster statistical significance at higher costs Works in closed environments where auction process is fully visible, e.g. owned inventory like social channels Open environments, offering highest level of precision and scalability across acquisition, retention and retargeting
ProsSimple setup; not cookie dependent More reliable than ITT, creating a more controlled test environment with both groups seeing ads Avoids costs for non-branded ads; can effectively compare exposed users with those who would have been exposed Most accurate method for open web advertising; significantly reducing noise; scalable across the funnel
ConsNoisier due to some treatment group users not seeing adsCosts to serve non-branded PSAs; results may be biased Limited to closed environments and cannot be used on the open web; cookie dependent Complex setup; cookie dependent

Maximize incrementality testing on the open web 

Ghost Bidding is the most advanced and rigorous methodology available for open web advertising. Ghost Bidding is adapted from ghost ads approach used in closed environments but is designed for the open web.  

In ghost ads, a false bid is placed that never actually produces an ad. The advertiser then compares the treatment group with users who would have seen an ad if the bid had been real. However, Ghost Ads can only be used in closed environments where the entire auction process is visible.  

Ghost Bidding takes a similar approach, but is optimized for the open web. A bid is placed as usual for the treatment group, while the control group is tracked to determine if the bid would have won the auction. If so, a “ghost impression” is recorded for the control group user. This allows for an accurate comparison of exposed users to those who would have been exposed, without the limitations of ghost ads in an open environment.  

At Criteo, we leverage a Ghost Bidding approach using a random split where only users with a high probability of being reached are analyzed.  

Our methodology offers several benefits: 

  • Significantly reduces noise for more accurate results by focusing only on users with a high probability of being reached  
  • Can be applied with an always-on approach across acquisition, retention, and retargeting tactics focused on conversion 
  • Like Ghost Ads, maintains a true control group of users that are not exposed to any ads 

Getting started 

Incrementality is a vital measurement method for understanding the true value of your advertising campaigns. Carefully selecting the right methodology and ensuring proper test setup will help you generate actionable insights, optimized strategies, and more informed budget decisions.   

To enhance robustness and impartiality of your incrementality testing, consider collaborating with third-party experts like Measured, Nielsen and Rockerbox. “We support and encourage partners to test incrementality with third-party measurement parties who bring deep expertise, industry best practices and unbiased perspectives.” Martin Froschauer, Senior Product Manager, Measurement. 

Get started today and talk to an expert. 

For further resources on incrementality, explore our success stories with French retailer, Gémo, and Publicis Groupe’s premier integrated agency, Digitas. 

See how targeted audiences drove better results

Discover why 19,000+ global clients trust Criteo as their preferred advertising partner.

190%
Increase in site visitors year-over-year (YoY)
+12%
Website visits from new prospects
Carmen Chan

Carmen is a Senior Product Marketer for Commerce Growth at Criteo. When she's not helping digital marketers drive business growth through performance advertising, you'll find her hosting gatherings with friends and family, or enjoying quality time with her two kids and husband. Originally from ...

Senior Product Marketer
Subscribe to our newsletter

Fresh sales trends and consumer insights to help you plan and win.