What is incrementality? How does it work? Find out why app marketers should start measuring conversion lift now. 

Proving the incrementality of a marketing channel is still one of the main challenges app marketers face today. But with fragmentation in measurement methodologies and no industry standard to calculate campaign lift, advertisers often struggle to answer a simple question: “Is this campaign generating conversions that wouldn’t have happened otherwise?”

#1 What is Incrementality?

Incrementality measures the impact of advertising efforts by comparing a Treatment group (users that can be exposed to the ad campaign) with a Control group (users that don’t see the brand’s ad) to calculate the lift in conversion in an apples to apples comparison. 

Incrementality, therefore, refers to the measurement of lift, the percentage of increase or decrease of a performance metric when comparing the Treatment group with the Control group.

#2 What are the benefits of measuring incrementality? Should I measure incrementality on app marketing?

  • Unlock performance insights: Lift measurement allows marketers to assess whether growth partners are actually adding value or just generating events that would have happened organically. Continuous lift measurement (vs isolated tests) allows marketers to track long-term conversion patterns instead of one-offs.
  • Ensure budget optimization: Focus on the campaigns that deliver incremental revenue with statistical significance in order to maximize long-term growth.
  • Set the right incentive for growth patterns: For example, when working on a CPA basis, the incentive for DSPs is set on reaching users that are lower down the conversion funnel, and therefore more likely to interact with an ad. As a consequence, users outside of that stage might not be leveraged to the fullest 👎. When having campaign lift as a KPI, however, growth partners focus on generating incremental revenue no matter the stage of the funnel. 

#3 How often should I test incrementality?

Continuously! Don’t think of lift analysis as a one-hit-wonder. A successful app marketing strategy should be measuring incrementality non-stop and non-disruptively. 

In fact, an isolated incrementality test is not the most efficient alternative. Traditional incrementality tests require advertisers to temporarily pause marketing efforts for the duration of the test, which in turn:

  • disrupts campaign performance due to constantly having to pause and restart campaigns, making it hard to ramp up to the previous level.
  • offers limited results, since one-time tests provide static incrementality “snapshots” that don’t reflect external factors that might influence campaign performance, such as seasonalities, special dates, or promotions. 
  • doesn’t reflect the real context in which campaigns normally run. Campaigns always run in parallel with other channels. To really understand the incrementality of a programmatic app campaign with a certain partner, marketers should be able to measure the lift of that specific campaign on top of all the others rather than having to exclude other channels. 

This is why we encourage marketers to measure lift continuously, rather than conducting periodic tests. 

#4 How does incrementality measurement work for App User Acquisition and App Retargeting?

Wait, is this even possible? 😱 Absolutely. While most advertisers will have come across some lift test or experiment for App Retargeting, lift analysis can also be applied to User Acquisition. As a matter of fact, apps with high brand awareness and organic traction should be measuring lift for User Acquisition. Simply put, it’s the only way to ensure your growth partners aren’t cannibalizing organics, but rather providing real value by finding new users for your app. 

So, back to the question of how… 

In most methodologies, the user base (also known as the population) is divided into the Treatment group, which could be exposed to the advertiser ads, and the Control or Holdout group, which doesn’t. This is done to then compare the conversions between the two groups. That said, there are some differences worth noting. For example: some methodologies actually serve ads to the Control group 👀. Let’s take a closer look: 

#5 What is a good conversion lift?

This is a question we receive quite frequently—and a tricky one. 

Is the answer eleven?

There is no universal answer. 

Advertisers don’t love that answer but it’s true. In the same way that conversion rates and CPAs vary per app vertical and type of campaign, so does incrementality.

If you are measuring incrementality continuously with a solid methodology and the lift is positive, you are already ahead of the curve. 

Always-on Lift Measurement 

Jampp’s Always-On Lift Measurement feature allows app marketers to measure incrementality across the entire app growth cycle: from User Acquisition to App Retargeting.

We use the scientific A/B test format (random split between Treatment and Control groups) and leverage the power of Ghost Bids. 

  • To eliminate any potential bias without adding unnecessary noise, we randomize the sample based on a single, random character of every device ID we see on RTB. 
  • When we get a bid opportunity for an eligible user, we make sure to bid only if the user is part of the Treatment group. This is how we guarantee an apples-to-apples comparison: both users could have been exposed to the ad if we had placed the bid. 
  • Finally, we leverage synthetic impressions to make results more accurate by eliminating aggregated data from users who were served ads but didn’t see them. To do so, we replicate the win rate of the Treatment group in the Control group. The random holdout users will then have the same distribution of impressions as the Treatment group; and, therefore, the same opportunities of being attributed.

Measuring the impact of advertising efforts should not be expensive or disruptive. Jampp’s Always-on Lift Measurement offers full transparency into the campaign’s incremental performance, for growth marketers to identify top strategies head-on and optimize based on success metrics. 

Wrapping up

Is there anything else you’d like to know about incrementality? Any unanswered questions about lift measurement keeping you up at night? Wondering if you qualify to try our Always-on Lift Measurement Feature? Send us a note or request a demo to take our Always-on measurement tool for a spin.