This post is for risk managers who want to explore tackling underwriting challenges through an Artificial Intelligence or Machine Learning (ML) toolbox. Here we examine two of the most popular tools for assessing the accuracy of a credit scoring models; Area Under the Curve (AUC) and Gini coefficient (Gini). We will briefly explain what AUC and Gini mean and how to interpret them in the context of credit scoring.
Area Under the Curve & Gini Coefficient
There are two primary questions in the lending industry:
- How risky is a customer?
- Should we lend to a borrower given his or her risk?
Image credit: Trae Gould
The answer to the first question is related to KYC assesses the likelihood of a customer defaulting and the probability of an application being fraudulent. Establishing the ability of a customer to repay determines the borrowing capacity of a customer. At the same time, this question also sets the interest rate and the term the borrower should have, as the interest rate measures the riskiness of the borrower, among other things (such as time value of money).
For example, the riskier the borrower, the higher the interest rate. With interest rate in mind, we can then determine if the borrower is eligible for the loan using a credit scoring model. The second question relates to the probability of a customer repaying given the loan amount, interest rate and repayment schedule. It is crucial to consider whether the loan will be profitable, even if a customer’s payments are made in a timely fashion.
Area Under the Curve
Both Gini and AUC are standard measures of accuracy for assessing the performance of credit scoring models. Both of these measures can be used in similar manners and AUC is needed to establish Gini. Gini is, in fact, a simplified representation of AUC, as Gini equals AUC * 2 - 1.
The AUC is calculated as a number between 0.5 and 1. It plots the relationship between true positives and true negatives. A true positive in credit risk assessment is a measure of how many creditworthy applicants are correctly identified as creditworthy. Whereas, true negative is a measure of how many uncreditworthy applications are identified as uncreditworthy. A good model will result in a high number of true positives and true negatives. The higher the AUC, the better the model is at assessing creditworthiness. Gini and AUC are intrinsically related as Gini is derived from AUC.
A Gini coefficient can be used to evaluate the performance of a classifier. A classifier is a model that identifies to which class or category a request belongs to. In credit risk, classifiers can identify if an applicant belongs to the creditworthy or the uncreditworthy categories .
Gini is most commonly used for imbalanced datasets where the probability alone makes it difficult to predict an outcome. The Gini coefficient is a standard metric in risk assessment because the likelihood of default is relatively low. In the consumer finance industry, Gini can assess the accuracy of a prediction around whether a loan applicant will repay or default.
Gini is measured in values between 0 and 1, where a score of 1 means that the model is 100% accurate in predicting the outcome. A score of 1 only exists in theory. In practice, the closer the Gini is to 1, the better. Whereas, a Gini score equal to 0 means the model is entirely inaccurate. To achieve a score of 0, the model would have to ascribe random values to every prediction.
A higher Gini is beneficial to the bottom line because requests can be assessed more accurately, which means acceptance can be increased and at less risk.
In the above example, the lighter colour below the darker lines is the AUC. The Gini here is calculated as follows:
Client Gini coefficient
= AUC*2 - 1
= (0.77 * 2) - 1
How are AUC and Gini scores used
Risk scoring models anticipate a person's probability of defaulting by assessing creditworthiness. Credit scoring models provide the variables for risk calculations. Calculating the risk involved in providing finance to a customer is essential for helping consumer finance organisations decide whom to grant credit to, how much to award, what the interest rate should be, and how to increase their bottom line. The scores obtained from credit scoring models are used for decision-making. As such, ensuring model accuracy is critical for both.
How does Instantor use Gini and AUC scores?
Predictive models classify loan applicants into two categories: good and bad. The good classification applies to an applicant with a low probability of default, and the bad classification applies to an applicant with a high probability of defaulting. Our clients’ acceptable limits are individual and based on the amount of risk a consumer finance organisation is ready to accept.
The definition of a good and a bad applicant differs widely based on the business model, loan lifecycle and whether the consumer finance organisation has an in-house or outsourced collection. Instantor considers this and accordingly adapts to the individual definition of each client.
The same applicant can be classified differently depending on the loan amount requested, the repayment schedule and the interest rate, at the same time, the financial situation of the borrower remains constant. Here is where Instantor makes it possible to combine all our data with the known facts to make more accurate decisions and at less risk. With tailor-made classification models, consumer finance organisations can tune loan features to lower the risk and increase profitability.
Instantor uses Gini and AUC scores to measure the performance of our models and the predictive power of our features. Instantor’s latest product, Insight, can improve Gini by up to 15 percentage points compared to using only traditional scoring data. Here, you can view how Insight’s new features increase both Gini and AUC for new and recurring customers, offering better predictions to help you reduce credit losses by up to 25%.
For several years, we at Instantor have helped consumer finance organisations make the most profitable decisions through our digitalised financing processes. We understand that your main challenges are in day-to-day operations: limited loan acceptance, fraud, bad loans, and default, and consequently, a reduced profit. In this article, we will dive into calculating risk to demonstrate how Instantor is helping clients reduce credit losses by a quarter.
 A classifier is an algorithm that makes a prediction about something, this could be predicting whether or not someone will get cancer, or it could predict if a loan will be good or bad.