When a national commercial bank in China ran a mobile phone app campaign, downloads reached 24 million. One of the main KPIs of the campaign was to encourage users to link their credit cards to the mobile app to promote more frequent use. Unfortunately, only 13 million users linked their cards.
The bank planned to launch another campaign to encourage users to link their cards to the mobile app. However, with a team of 100 telemarketers that could be used to make 200 calls per day contacting the the 13 million users that did not link their cards, it would take nearly two years to reach out to everyone.
The bank needed to find a targeted group of customers that would be more responsive to such a campaign for the telemarketing team. How would they find this target group in a short amount of time?
They asked their IT department’s data analytics manager and his team to use their existing machine learning tools to find a solution, as they can be used to help make predictions from existing data. The team tried to train, evaluate and deploy a model that could pinpoint a targeted, highly responsive group of customers. However, they found their existing software solution was not user-friendly. It required model-building experience to create these highly accurate predictions. The process also required experience in optimizing and making the models scalable.
How SPM Helped
QY Datatech Inc, Minitab’s Authorized Partner in China, introduced the bank to Salford Predictive Modeler (SPM). The team used two of SPM’s modeling engines, CART® and TreeNet®, which were particularly powerful when faced with larger and highly complex datasets with diverse variables. In this case, the variables were a combination of a customer’s details and other data – such as the frequency of the customer using the bank’s app.
For example, the CART modeling engine produced a single decision classification tree that took categorical data to predict a qualitative value and historical data that could be segmented into a set of yes/no rules. This segmentation splits the response (Y) variable into partitions based on the predictor (X) settings. Continually growing or “pruning” the CART tree helped the team quickly identify additional causes of the excessive variability in this process. Once the team had narrowed down to a few vital predictor variables, controls were put in place to limit the results to customer groups with specific rules such as customers 25 years old or over or having an annual income that was more than USD 40,000. These rules were related to targeting the demographics of customers that would more likely link the mobile app to their bank account.
The team found the CART model showed some hidden gems, such as the customer group with specific rules, which can be difficult to discover with traditional queries generated using regression and ANOVA. Each split on the graph above shows that data is segmented into two groups based on the value of one predicting variable.
The partial dependency plots in TreeNet explains the reaction between one variable with a target in an intuitive way.
The team then used the TreeNet modeling engine to quickly narrow down the variables that made the most impact and could be used to illustrate the relationship between the variable and the outcome. The outcome was the chance that customers would link the app with their bank account and the identified variables in this example included monthly income shown in the graph above.
The data analytics manager used Gain chart in TreeNet to evaluate the model. TreeNet shows gain/lift statistics by truncating training data (bagging) into several segments. The user may also choose to analyze training/testing or pooled bagged data with various other measurements, such as the Receiver Operating Characteristic or ROC. The ROC curve is an important evaluation for binary prediction. The higher the ROC is, the better the model performance is.
TreeNet has a summary window that displays some critical statistics. This plot shows the ‘negative log likelihood’, a statistic measurement for model checking/comparison that emphasizes the probability interpretation of model predictions.
Through training, testing, and eliminating low-response customers from the list and then optimizing and validating the models, the team was able to find a targeted group of customers. The models generated were able to identify characteristics of the “highly responsive” customer group by scoring and evaluating each customer’s response rate based on the customers’ historical responsiveness to bank promotions in the past year.
The team at the bank said the best part was that SPM was easy to use. With only a few clicks and without the need to code, they were able to generate initial models and reduce the overall modeling time from months to days. In fact, it only took seven days to train and deploy a refined model that was able to predict the target group.
With this refined group of 2 million app users to contact compared to the initial 13 million, the telemarketing team was able to reach out to everyone within a few months. As a result, they saw a 300% increase in users linking their cards to the mobile app. The bank also avoided the cost associated with recruiting additional telemarketers and increased efficiency and promoted these revenue-generating services to targeted customers at the same time. The telemarketing team also reported a 35% higher success rate in encouraging the users to link their card compared to previous campaigns. Armed with Salford Predictive Modeler, the project team was able to help the bank understand and use their data to generate fact-based insights to make cost-saving decisions.