A leading Australian telco were undertaking a raft of customer retention initiatives and engaged White Box to build a predictive model to evaluate the churn risk posed by its customer base.

In an extremely competitive business environment, reducing churn numbers, even by small percentages, has the potential to make a huge positive impact on the company’s bottom line. By identifying the features that impact churn most, the business will also be better placed to deliver value to customers who may be dissatisfied or disengaged, in turn increasing loyalty and brand reputation.

Requirements

The key requirements of the project included to:

Clearly define churn in the company’s specific business context
Build a churn propensity model
Create a process for the propensity scores to be refreshed frequently
Provide supporting customer data to help inform the nature of the score, e.g. top important features
Workshops with the wider teams to explore a customer offer matrix
Provide appropriate technical specification documents

Methodology

The project commenced by working closely with the client across multiple functions of the business including its Data & Analytics (DnA), Customer Relationship Management (CRM) and Customer Service Centre (CSC) teams. This allowed for the project methodology to be mapped, including defining what churn actually is, who we were predicting to churn, and what we can already extract about those customers in order to start testing the most suitable model.

Defining churn

SOFT CHURN

Customers who fail to renew their plan 10 or more days after the plan expiry date

HARD CHURN

Customers who end their service explicitly, by cancelling, disconnecting or porting out

The above may look a little confusing but essentially it allowed us to spot both forms of churn and use them to evaluate a consolidated churn value for every customer.

Population

Cohorts of the telco’s customer base over the last two years were used in order to build an authentically representative dataset, in particular to avoid COVID bias. The active customer base as at each of the chosen cohort dates was selected with features evaluated historically.

Long expiry customers, and customers already more than 10 days into soft churn status at the cohort date, were excluded.

Features

These are just a few examples of the features used in the model. Different selected features are weighted within the model according to their ability to more effectively predict churn, of which you will see the successfulness below.

We built and tested ~200 features throughout the process.

Model selection

There are many modelling techniques available to the data scientist, and a range were tested and evaluated based on their overall predictive power. The results are represented by the coloured lines in the chart below, but we won’t go into the science here. Rather, to simplify, the grey diagonal can be thought of as a coin toss (a 50% chance of correctly predicting a positive result regardless of the prediction parameters we use), and the cross at the top left an impossibly perfect model (a 100% chance of always correctly predicting a positive result). In between are some of the actual results, with the objective being to find the model that gets us closest to perfection.

Decision trees, logistic regression, and random forest techniques were tested amongst others, with random forest (indicated by the light blue line) the best performer.

Results

After going through the process of running many iterations of the model to tune parameters, eliminate weak predictive features, and observe diagnostics, the final model had an accuracy score of 96%, i.e. correctly predicted that 96% of customers would either churn or not churn (quadrants A + D in the diagram below).

Whilst high accuracy is a pleasing result, it’s not the decisive factor in determining the model’s effectiveness. Other measures known as precision and recall give us a clearer picture of the value of the model.

Precision is a measure of how many predicted churners actually churned. While we seek a value as high as possible, something in the range of 70-90% is a realistic goal. The client will invest time and marketing spend on predicted churners, so a high value here indicates that those resources haven’t been wasted on people who were not in fact intending to leave.

Recall is a measure of how many actual churners were predicted to churn, with a similar objective range to precision. If recall is 75% it means one quarter of all churners were missed, an opportunity lost. However three quarters of churners were captured, with the potential to deliver huge revenue gains. Even low recall rates are typically better than the scattergun approach to churn often employed by companies.

Feature importance

With predictive modelling our end goal is to produce a score for each customer that encapsulates all the model inputs into a single actionable value. However, we’re always interested to know what customer features have most influenced those values at a business-wide level.

It’s important to note that we’re not looking for surprises here - the more explicable the results of feature importance, the more confidence we can have in the model. The magnitude and ordering of features though, usually provides some interesting insights and can challenge preconceptions of what matters most to customers.

Below is a generalised feature importance chart from the model, with the actual features masked for privacy purposes.

Scoring & segmentation

The final stage of the model build is to score and segment the customer base. Each customer is assigned a precise churn propensity score, but a higher level score is also typically applied based on observable customer segments. The segmented scoring system used can be tailored to the intended marketing strategies to be deployed, and should support straightforward and fast decision-making by non-scientific users.

This is a powerful visual which effectively categorises customers of the telco by their risk of churning. From a marketing perspective, it gives an indication of which customer group to target in order to increase the retention rate of customers by the most significant amount within budget parameters.

It makes more sense to target the Persuadables, who are moderately likely churners but are not already out the door (Lost causes), rather than those with a lower risk (Do not disturbs and Sure things), who are unlikely to churn anyway. This means your marketing spend is allocated to the customers who need your attention most!

Conclusion

This churn project will significantly improve the telco’s bottom line if their customer service and marketing tactics are employed effectively. By having a model which scores each customer and identifies the most important features that contribute to churn, the telco’s teams can more efficiently focus their efforts to communicate with and service its customer base.

Want to know how your business could implement a similar solution? Contact us now to find out more.

Get in touch