Skip to main content
All CollectionsManaged Services: Predict & Experiment
Understanding Estimated Controls
Understanding Estimated Controls

Learn how Persado calculates lift in the absence of a control in market.

A
Written by Amy Blakemore
Updated over 9 months ago

Sometimes, you may not always have a control message in market. Common scenarios we see where this occurs are:

  • You want your full audience to receive the optimized message

  • You don’t have enough volume to measure a Variant against the control with significance

  • You don’t have a control to begin with

  • You simply prefer the ease of deploying just one Variant.

So, how do we properly calculate our impact? An estimated control is the estimated performance of a control message that our Data Science team calculates in the absence of a control message in market. This article will outline why we calculate these and how we do it.

The Importance of Estimated Controls

It’s critical that our Data Science team calculates estimated controls so we have an accurate measure of how much impact Persado generates in market. This is done in the form of incremental responses, and more importantly, incremental revenue. Ultimately, we want to make sure you are able to confidently report our achievements to internal stakeholders.

How Estimated Controls are Calculated

Typically, we calculate estimated control performance by using a certain lift and working backwards to calculate the control response rates. For example, if there was a 15% lift in Exploration, but no control in Broadcast, we would estimate the Broadcast control's performance by calculating what numbers would lead to a 15% lift in Broadcast. Our Data Science team calculates this using proxy analyses, or analyses based on past results that have the highest degree of similarity to your current deployment, and that had a control in market. Proxy analyses always come from the same channel.

For example, if we are in need of an estimated control for a Predictive Content subject line deployment, we would follow these steps:

  1. First, we select up to 20 past Experiments that had Controls deployed, prioritized by similarity to the current Experiment, and by recency.

  2. Second, we calculate a weighted average lift from those proxies.

  3. Third, we take that lift and the current Experiment's Persado response rate, and use those to calculate what the estimated control response rate should be to match that lift.

Need a refresher on common reporting KPIs? Check out this article to better understand lift, elasticity, and more.

Did this answer your question?