Personal Credit Scoring

Personal Credit Scoring

1. tech- niques

   •  help organizations decide whether or not to grant credit to consumers who apply to them    
   • credit scoring
     •  determines whether the applicants is qualified    
     • problem of classification
       • purpose is to make a distinction between “good” and “bad” custom- ers.    
       •    The banks should extend credit to “good” ones in order to increase the revenue and reject “bad” ones to avoid economic losses    
     • History and development
       • 1941 & one of the first examples of expert systems
         • David Durand [2] was the first to recognize ones should differentiate between good or bad loans by measurements of the applicants’ characteristics    
         •     credit analysts in financial companies and mail order firms decide to whether to give loans or send merchan- dise.    
         • rules were summarized and then used by non-experts to help make credit decisions    
       • arrival of credit cards in the late 1960s
         • the banks and other credit card issuers begin to employ credit scoring.    
         •     credit scoring not only improves the forecast ac- curacy but also decreases default rates by 50% or more.    
       • 1970s, completely acceptance of credit scoring
         •       leads to a significant increase in the number of profes- sional credit scoring analysis.    
       • 1980s, credit scor- ing has been applied to personal loans, home loans, small business loans and other fields.
       • In the 1990s, scorecards were introduced to credit scoring
     • three basic techniques
       • scoring models
       • statistical models
       • artificial intelligence (AI) methods
   • behavioral scoring
     •                  how to deal with exist- ing customers, such as should the firm agree to increase his credit limit or what actions it will take if the customer starts to fall behind in his repayments    

2. Overview of the Techniques

   •  Statistical models and AI models are two most important methods for credit scoring                 
   • Statistical Model
     • LDA (linear discriminate analysis)
       • LDA uses linear discriminate function (LDF) which passes through the centroids of the two classes to classify the customers. =      analysis whether the user’s characteristic variables are correlation    
       • most widely used sta- tistical methods for credit scoring
         • Limitation
           • However been criticized for its requirement of linear relationships between dependent variables and independent variables and the assumptions that the input variables must follow Normal distribution.    
       • Logistic regression
         • To overcome these drawbacks, Lo- gistic regression, a model which not requires the normal distribution of variables, was introduced.    
         • ability to predict default probability of an applicant and indentify the variables related to his behavior
           • LR is proofed as effective and accurate as LDA, but does not require input variable to follow normal distribution.    
     • MARS (multivariate adaptive regression splines)
       • non-linear and non-parametric regression method first proposed by Friedman    
       •       has strong generalization ability and excels at dealing with high-dimensional data    
       • optimal MARS model
         • First Stage
           • very large number of basis fuctions are constructed to over fit the data, which can be continuous, categorical and ordinal.    
         • second stage
           • basis functions are deleted in the order of least contribu- tions using the generalized cross-validation (GCV) crite- rion    
           •   A measure of variable importance can be assessed by observing the decrease in the calculated GCV when a variable is removed from the model. This process will continue until the remaining basis functions all satisfying the pre-determined requirements.    
       • superior to artificial intelligence
         •      short training time and strong intelligibility    
       • usually used as a feature selection technique for classifier in hybrid models in order to ob- tain the most appropriate input variables
     • Bayesian Model
       •       predict class membership probabilities, such as the probability that a given sample belongs to a particular class    
       • Naive Bayes
         •             assumes that the attributions of a sample are independent. Although it simplifies the calculation, the variables may be correlated in reality.    
       • Bayesian belief networks
         • graphical model and allow class conditional independen- cies to be defined between subsets of variables.    
         • Bayesian networks
         • directed acyclic graph
         • conditional probability tables
     • Decision Tree
       • recursive partitioning    
       • customer data are divided into limited subsets to make the homogeneity of default risk in the subset higher than original sets.    
       • division process con- tinues until the new subsets meet the requirements of the end node.    
       • REPTREE CLASSIFIERS FOR CREDIT RISK FORECAST
       • A Comparative Study of Reduced Error Pruning Method in Decision Tree Algorithms
       • bifurcation rules
         •  used to divide new sub sets    
       • stopping rules
         • determine that the subset is whether or not an end node    
       • rules deciding which class the end node belongs to
     • Markov Model
       • Markov model uses history data to predict the distribu- tion of population in a time point at regular intervals.    
       • speculates the trend of a population based on the regula- tion of population changing in the past    
   • Artificial Intelligence Methods
     • LDA and LR are two effective statistic methods. How- ever, Thomas [1] indicated that the classification accu- racy of the two models was not very high.    
     •   more and more Artifi- cial Intelligence methods are applied for credit scoring, such as artificial neural networks (ANN), genetic algo- rithm (GA) and support vector machine (SVM    
     • Artificial Neural Networks
       • information processing model resembling connections structure in the synapses. It consists of a large number of nodes (also called neurons or units) by links.    
       •     The feed-forward neural network with back-propagation (BP) is widely used for credit scoring, where the neurons receive signals from pre-layer and output them to the next layers without feedback    
       • Back-propagation
         •  learns by iteratively processing a set of training samples, comparing the network’s prediction for each sample with the actual known class label.    
         •       For each training sample, the weights are modified so as to minimize the mean squared error between the network’s prediction and the actual class. The modifications are made in the “backwards” direction, so we name the net- work back-propagation.    
       • Various activation functions
         • RBF is the most common activation function. It is non-negative and non-linear, a center symmetrical attenuation local distri- bution function.    
         •       There are three parameters in RBF: the center, variance and weight of the input units. The weights are obtained by solving a linear equation or using least square method recursion, which makes the learning speed faster and void the problem of local minimum point.    
       • Advantages of neural networks include their strong learning ability and no assumptions about the relation- ship between input variables.
       • drawbacks
         • neural networks lies in their poor understandability    
         • it is very difficult for ANN to make knowl- edge representation
         • design and optimize the network topology
           •  It is obvious that, the amount of units and layers in hidden layer, different ac- tivation function and initial weight values may affect the final classification result    
         • needs a large number of training samples and long learning time.
           •  Desai, Crook & Overstreet [12] made a comparison of neural networks and linear scoring models in the credit union environment and the results indicated that neural network had better performance for correctly classifying bad loans than LR model.    
     • SVM (support vector machine)
       •       SVM is suitable for small samples and doesn’t limit the distribution of data    
       •   it is based on structural risk minimization (SRM) rules which ensure a good ro- bustness theoretically.    
       •       The principle of SVM classification is to find a maximal margin hy- perplane to separate examples of opposite labels.    
     • Genetic Algorithm and Genetic Programming
       • Genetic Algorithm
         • computing model which simulates natural selection of Darwinian biological evo- lution theory and the process of biological evolution in genetic mechanism with the purpose of searching the optimal solution.    
         • main principle:  According to the evolution theory of “survival of the fittest” and beginning from the initial generation popula- tion, some genetic operations including selection, cross- over and mutation are applied to screen individuals and generate new populations visa pre-determined fitness function. 
         • This process continues until the fitness functions achieve greatest value and obtain final optimal solution.                  
       • Genetic Programming
         •        The basic theory of GP is the same as GA.    
         • each generation of individuals are organized through a dynamic tree structure    
     • K-Nearest Neighbor
       •  The training samples are described by n-dimen- sional numeric attributes. Each sample represents a point in an n-dimensional space.    
       •  When given an unknown sample, a k-nearest neighbor classifier searches the pat- tern space for the k training samples that are closest to the unknown sample.    
   • Hybrid Models
     •  synthesizing advantages of various methods have become hot research topics    
     •        there is not a clear solution to how to classify- ing the hybrid models.    
     • Hybrid method for assessing credit risk using Kolmogorove-Smirnov test, Fuzzy Expert system and DEMATEL method were suggested in            Sanaz Pourdarab, Ahmad Nadali and Hamid Eslami Nosratabadi. 2011. A Hybrid Method for Credit Risk Assessment of Bank Customers. International Journal of Trade, Economics and Finance, Vol. 2, No. 2, (April 2011)    
     • Simple Hybrid Models
       • pro- cess of building a credit evolution model into three steps: feature selection, determination of model parameters and classification. Simple hybrid approach means choosing different methods in these three stages.    
     • Class-Wise Classifier
       •        The information and data collected by credit institutions are often not complete. One solution is to cluster the samples automatically and decide the number of labels before classification stage.    
     • Ensemble Classifier
       •     First, several classifiers are produced and obtain classification results by training them on different samples. Then choose the right classifiers as ensemble members according to certain criterions. Finally, aggre- gate these ensemble members visa ensemble approach and get the ensemble result. Boosting, stacking and bag- ging are often used as ensemble approaches. Nowadays, ensemble learning has become the latest method of credit evaluation model.    
       •       deal with the issues of missing data and imbalance classes and perform better classification ability and per- dition accuracy compared with signal or simple hybrid models.