The Intuition Behind Uplift Modeling

Businesses pay a lot of attention to the estimated return on investment from the marketing efforts (e.g. promotions, communications) they employ. Insights into the impact of marketing on their customers are critical to optimizing ROI. A/B testing is an approach well-understood and commonly applied to measure the impact of marketing on the affected (treatment) group relative to an unaffected (control) group. While A/B testing is informative, it is limited to measuring the lift or additional sales for the whole group of treated customers on average. Using uplift modeling techniques, we can take A/B testing a step further to make more precise measurements of lift at the customer level. 

[Related article: Why Causation Matters in Data Science]

To best explain the concept of uplift modeling, imagine a business wants to drive sales by offering 15% off. The business hopes that the discount will incentivize people to spend enough that the additional sales will offset the profit lost from the discount. Because the business wants to measure the performance of the sale, they keep a random group of customers from getting the discount (control group). At the end of the sale, people who were offered the discount (treatment group) spent on average $100 per person, and the control group spent on average $60 per person. Forty dollars of lift on average, not bad! Now, some of those customers were going to buy shoes regardless of whether the business offered the discount, so the business lost the 15% off on those “Sure Thing” customers. Some of the customers weren’t going to spend during the sale no matter what, so the business lost revenue on the dollars spent executing the discount process for those “Lost Causes” customers (mostly communication expenses probably). The only customers the business wants to give 15% off to, are those that will purchase more only as a result of getting the discount, or the “persuadables”. To identify these customers, we need a statistical model to estimate the difference in sales expected given a customer got the discount instead of not receiving the discount. 

To identify the “persuadables” that offer the highest lift, we need to predict something we can never truly observe, the difference in the behavior of a customer under two different environments (treatment or control) simultaneously. So, we don’t have a column of observed additional sales dollars to fit a model on. Instead, we’ll model the sales generated by customers in the two different environments (treatment and control) and use the model(s) to infer the lift, or the additional sales generated by the treatment. We can simply take the difference between sales expected under the treatment and those expected under the control. 

Lift = Expected_Sales|Treatment – Expected_Sales|Control

The most intuitive modeling approach to estimating uplift modeling is the use of two separate models. One model is built to predict sales for customers that received the treatment, and another model for those that received the control. We can use any type of supervised learning algorithm, random forest, boosted regression, linear regression, etc. To calculate the expected lift for a customer we can simply take the difference in predicted sales values from the treatment and control models. Using our previous example of the 15% off discount, imagine the customers that were female and hadn’t shopped at the store before spent only $20 under both treatment and control scenarios. We would expect $0 off lift, because the models should both predict $20 of sales. Let’s say male customers who were high spending individuals also spend about the same under treatment and control scenarios; $0 of lift for them as well. However, moderate spending male customers spent $200 under the treatment and $100 under the control. We would expect $100 of lift from those customers. As a refresher, on average our 15% off discount generated $40 of lift. So, next time we issue that discount, we would profit the most by only offering it to those moderate spending male customers, for which we had the highest lift. 

This article is meant to be an introduction to the concept of uplift modeling, which requires a considerably different approach than typical supervised learning. We barely scratched the surface of the technical challenges associated with estimating uplift and the use of two models is the simplest of many modeling techniques that can be applied. I highly recommend looking into the “Pylift” library for Python users, and the “Uplift” package for R users. 

Editor’s note: Want to learn more about uplift modeling? Check out the upcoming ODSC West talk on the topic, “Uplift Modeling Tutorial: From Predictive to Prescriptive Analytics,” by Victor Lo, PhD & Head of Data Science & Artificial Intelligence at Fidelity Investments this October!